[PR #5524] [MERGED] perf(go): summarize build identity and query retained metadata #6951

New issue

Closed

opened 2026-11-01 21:44:17 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:44:17 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5524
Author: @gensokyo-dev
Created: 10/20/2026
Status: ✅ Merged
Merged: 10/20/2026
Merged by: @gensokyo-dev

Base: main ← Head: perf/5499-go-metadata-views-20261020

📝 Commits (4)

1b02e02 perf(go): summarize build identity and query retained metadata
ceb2666 fix(experiments): finalize Go receipts after transcript failures
3b2c7ad fix(go): reject conflicting parent architecture in retained views
99003cf docs(experiments): record latest-main Go qualification

📊 Changes

27 files changed (+3212 additions, -59 deletions)

View changed files

📝 docs/go-binaries.md (+50 -0)
➕ experiments/analysis-efficiency/go-build-view-costs.py (+74 -0)
➕ experiments/analysis-efficiency/go-build-view-oracle.py (+81 -0)
➕ experiments/analysis-efficiency/go-build-views.json (+1163 -0)
➕ experiments/analysis-efficiency/go-build-views.md (+195 -0)
➕ experiments/analysis-efficiency/go-build-views.mjs (+228 -0)
➕ experiments/analysis-efficiency/go-build-wire-costs.py (+86 -0)
➕ experiments/analysis-efficiency/go-build-wire.py (+292 -0)
📝 experiments/analysis-efficiency/retained-query-finalization.mjs (+5 -0)
📝 experiments/analysis-efficiency/test_retained_queries.py (+17 -0)
📝 src/application/go/GoBinaryPort.ts (+1 -1)
📝 src/application/go/GoBinaryService.test.ts (+20 -0)
📝 src/application/go/GoBinaryService.ts (+23 -16)
📝 src/cli/analysisViewCommands.ts (+1 -1)
📝 src/cli/goCommands.ts (+13 -7)
📝 src/contracts/analysisViewToolContracts.ts (+1 -1)
📝 src/contracts/go/goToolContracts.ts (+7 -3)
📝 src/domain/analysisView/analysisView.ts (+27 -0)
➕ src/domain/analysisView/goBinaryView.test.ts (+128 -0)
➕ src/domain/analysisView/goBinaryView.ts (+312 -0)

...and 7 more files

📄 Description

Summary

Fixes #5499. A Go compiler/main-module question currently receives raw encodings and every dependency. Add opt-in summary delivery and selected retained metadata views through the shared application/domain layer. Default complete replies and their Evidence identity remain unchanged.

Problem and expected behavior

GitHub CLI v2.102.0 produces 63,544 bytes of complete Go Evidence with 165 dependencies. A summary returns embedded identity, source ranges, counts and unknowns; dependency/settings pages select exact paths/keys, original indices and fields without reopening the target. Every row keeps its original index and parent identity. Complete bytes remain accessible in the retained parent or a CLI Evidence bundle.

Change and scope

The existing Go reader still owns all parsing. Shared views validate immutable parent facts once for their Evidence lifetime; mutable inputs are revalidated. CLI uses the existing non-overwriting bundle publisher; MCP retains the complete parent before projection. Scope decision: not applicable to this projection of existing read-only metadata.

Contract and boundary impact

Owner: Go domain facts and shared retained-view projection; no provider protocol or parser changes.
CLI/MCP: optional detail: summary; Go views support dependency/settings filtering, fields and paging. Default complete delivery stays compatible.
Evidence: unchanged complete parent; derived views carry identity, source ranges, coverage and explicit unknowns.
Target/authority: reads the selected stable file; CLI writes only the explicitly selected new Evidence output. Reader and publication keep their existing cleanup owners.
Packaging: no dependencies, installers, binaries or generated outputs committed. Catalogs are regenerated from contracts.

Expected effects:

Read existing files or captured data.
Launch or attach to a target process.
Inject instrumentation or run target scripts/code.
Write a caller-selected new Evidence output file for CLI summary delivery.
Contact network services during inspection.
Recover protected code or work around protections.

Evidence and regression coverage

Native go version -m, original byte ranges and before/after full exports agree for all 165 dependencies. Real CLI complete output is identical; summary bundles retain it, and file-based selected CLI queries match the native oracle. Fixtures cover duplicate paths, empty selections, field omissions, identity conflicts, unavailable metadata, mutable/malformed parents and bounded messages.

In the pinned pre-rebase cohort, actual Codex 0.163.0 with a deterministic local SSE backend captures every request, including repeated catalogs. No AI inference or paid model service is used. Whole captured-request JSON token proxies:

Question	Before → after
Compiler, main module and dependency count	158,529 → 123,368 (same two requests)
Identity plus one dependency	158,538 → 186,812 (two → three requests)
Same compound question plus export	256,342 → 250,352 (three → four requests)

Summary helps questions it directly answers; a follow-up round can cost more context. Repeated direct-SDK compound/export tasks also get slower: 4.855–4.952 → 6.370–6.564 s wall, 4.682–4.725 → 6.175–6.400 s CPU, 671–680 → 846–869 MiB family peak. Small tasks remain about 4.3 s; warm selected reads take 33 ms. No universal speed/memory gain or actual model usage is claimed. Methods, complete measurements and reproduction.

Observations, derivations and inferences remain distinguishable.
Artifact identity, provenance and failed attempts are preserved.
Missing metadata, partial parsing and runtime dependency completeness remain explicit.

Validation performed

Current head is rebased onto main@cf5194f68, retaining its new IL2CPP contract and the Go additions. All 258 focused tests in 12 files pass, including the 32 IL2CPP view tests and 13 CLI path tests. Full npm run check, npm run docs:check plus five README tests, and a rebuilt-runtime real SDK selected query/full export pass. The original complete parent identity and all 165 dependencies remain exact. Duplication passes at 3.49018% (baseline 3.49743%, gate 3.5%). The frozen timing cohorts below are not timings of this rebase.

After rebasing onto main@c374c2fe7: 10 files / 202 focused tests pass; npm run check and npm run docs:check pass (including five README consistency tests). A real rebuilt-runtime SDK selected query and complete export pass again with the original parent identity and all 165 dependencies. Eight frozen matched SDK tasks plus a warm diagnostic, six actual-client task variants, and real CLI complete/summary/bundle/file-view verification also pass. All qualified payload exits and owned cleanup are verified, with zero observed OOM events. Two qualification attempts stopped at the host-memory guard, including a latest-main build; owned scopes were verified inactive and unchanged-guard retries passed after headroom recovered. Final whole-scope counters are unavailable.

Review qualification: the shared benchmark finalizer now cleans owned resources before optional transcript publication. All 11 receipt-failure tests pass; a real selected SDK task/export passes, and an injected transcript-write failure exits 1 with a recorded failure and verified process-family cleanup.

Architecture review: Go domain now owns the existing report-to-subject architecture mapping. Production Evidence and every Go view share it, rejecting inconsistent inline/file parents, including conflicts between known and unknown architectures. All 213 focused tests and full npm run check pass after this correction.

The initial path-message/duplication failures disappeared on new main. Current CI shard 4 has four filteredFailureOutput.test.ts assertions expecting the old generic remediation text; all four reproduce on untouched main@cf5194f68 (npm run test:focused -- tests/boundary/cli/filteredFailureOutput.test.ts: 4 failed, 1 passed). Actual output uses the contract-aware recovery guidance added by main. The Go PR does not change that workflow. The control run used the pinned toolchain, bounded resources, verified cleanup and zero observed OOM events. Runtime/platform checks, including the Windows lane, passed. Matrix fail-fast cancelled shards 3 and 5; a direct shard-3 rerun was cancelled again because the original shard-4 failure remains. Their complete CI coverage is unverified; changed Go/Rust/CLI/MCP paths are covered by the 258 focused local tests and real SDK/CLI tasks. The owner explicitly authorized bypass for a verified PR blocked by unrelated CI; the bypass is for this reproduced baseline failure and its resulting aggregates/cancellations.

Compatibility, safety, and release

No default reply or complete Evidence identity change. Real verification covers Linux ELF; PE/Mach-O use authored fixtures. Metadata observes embedded claims and does not authenticate a compiler or establish complete runtime dependencies. Actual model reasoning, backend schema acceptance, rendering and billing are unmeasured. REA never executes an input. The installed Go tool is also invoked as the trusted native verifier.

Author readiness

Human owner: @gensokyo-dev

The owner explicitly dictated these declarations and initial readiness in the working session; the assistant entered them under that instruction.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved; no additional maintainer scope decision is required.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked.
Tests cover changed behavior and meaningful failure paths.
Owning docs, contracts and generated metadata are updated where needed.
Representative CLI/MCP output and measured limits are documented.
The final diff has been checked for secrets, unrelated cleanup and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5524 **Author:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Created:** 10/20/2026 **Status:** ✅ Merged **Merged:** 10/20/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `perf/5499-go-metadata-views-20261020` --- ### 📝 Commits (4) - [`1b02e02`](https://github.com/morluto/rea/commit/1b02e02e315ff44a8aa0b1e36f81782d28d6c63a) perf(go): summarize build identity and query retained metadata - [`ceb2666`](https://github.com/morluto/rea/commit/ceb2666579a00aa811485c7cfaa9846fe57e304f) fix(experiments): finalize Go receipts after transcript failures - [`3b2c7ad`](https://github.com/morluto/rea/commit/3b2c7ad410df211a6c3a9574ad5daeafa64a435f) fix(go): reject conflicting parent architecture in retained views - [`99003cf`](https://github.com/morluto/rea/commit/99003cf32fc77633cffa8434303837000e5ebcaf) docs(experiments): record latest-main Go qualification ### 📊 Changes **27 files changed** (+3212 additions, -59 deletions) <details> <summary>View changed files</summary> 📝 `docs/go-binaries.md` (+50 -0) ➕ `experiments/analysis-efficiency/go-build-view-costs.py` (+74 -0) ➕ `experiments/analysis-efficiency/go-build-view-oracle.py` (+81 -0) ➕ `experiments/analysis-efficiency/go-build-views.json` (+1163 -0) ➕ `experiments/analysis-efficiency/go-build-views.md` (+195 -0) ➕ `experiments/analysis-efficiency/go-build-views.mjs` (+228 -0) ➕ `experiments/analysis-efficiency/go-build-wire-costs.py` (+86 -0) ➕ `experiments/analysis-efficiency/go-build-wire.py` (+292 -0) 📝 `experiments/analysis-efficiency/retained-query-finalization.mjs` (+5 -0) 📝 `experiments/analysis-efficiency/test_retained_queries.py` (+17 -0) 📝 `src/application/go/GoBinaryPort.ts` (+1 -1) 📝 `src/application/go/GoBinaryService.test.ts` (+20 -0) 📝 `src/application/go/GoBinaryService.ts` (+23 -16) 📝 `src/cli/analysisViewCommands.ts` (+1 -1) 📝 `src/cli/goCommands.ts` (+13 -7) 📝 `src/contracts/analysisViewToolContracts.ts` (+1 -1) 📝 `src/contracts/go/goToolContracts.ts` (+7 -3) 📝 `src/domain/analysisView/analysisView.ts` (+27 -0) ➕ `src/domain/analysisView/goBinaryView.test.ts` (+128 -0) ➕ `src/domain/analysisView/goBinaryView.ts` (+312 -0) _...and 7 more files_ </details> ### 📄 Description ## Summary Fixes #5499. A Go compiler/main-module question currently receives raw encodings and every dependency. Add opt-in summary delivery and selected retained metadata views through the shared application/domain layer. Default complete replies and their Evidence identity remain unchanged. ## Problem and expected behavior GitHub CLI v2.102.0 produces 63,544 bytes of complete Go Evidence with 165 dependencies. A summary returns embedded identity, source ranges, counts and unknowns; dependency/settings pages select exact paths/keys, original indices and fields without reopening the target. Every row keeps its original index and parent identity. Complete bytes remain accessible in the retained parent or a CLI Evidence bundle. ## Change and scope The existing Go reader still owns all parsing. Shared views validate immutable parent facts once for their Evidence lifetime; mutable inputs are revalidated. CLI uses the existing non-overwriting bundle publisher; MCP retains the complete parent before projection. Scope decision: not applicable to this projection of existing read-only metadata. ## Contract and boundary impact - Owner: Go domain facts and shared retained-view projection; no provider protocol or parser changes. - CLI/MCP: optional `detail: summary`; Go views support dependency/settings filtering, fields and paging. Default complete delivery stays compatible. - Evidence: unchanged complete parent; derived views carry identity, source ranges, coverage and explicit unknowns. - Target/authority: reads the selected stable file; CLI writes only the explicitly selected new Evidence output. Reader and publication keep their existing cleanup owners. - Packaging: no dependencies, installers, binaries or generated outputs committed. Catalogs are regenerated from contracts. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a target process. - [ ] Inject instrumentation or run target scripts/code. - [x] Write a caller-selected new Evidence output file for CLI summary delivery. - [ ] Contact network services during inspection. - [ ] Recover protected code or work around protections. ## Evidence and regression coverage Native `go version -m`, original byte ranges and before/after full exports agree for all 165 dependencies. Real CLI complete output is identical; summary bundles retain it, and file-based selected CLI queries match the native oracle. Fixtures cover duplicate paths, empty selections, field omissions, identity conflicts, unavailable metadata, mutable/malformed parents and bounded messages. In the pinned pre-rebase cohort, actual Codex 0.163.0 with a deterministic local SSE backend captures every request, including repeated catalogs. No AI inference or paid model service is used. Whole captured-request JSON token proxies: | Question | Before → after | | --- | ---: | | Compiler, main module and dependency count | 158,529 → 123,368 (same two requests) | | Identity plus one dependency | 158,538 → 186,812 (two → three requests) | | Same compound question plus export | 256,342 → 250,352 (three → four requests) | Summary helps questions it directly answers; a follow-up round can cost more context. Repeated direct-SDK compound/export tasks also get slower: 4.855–4.952 → 6.370–6.564 s wall, 4.682–4.725 → 6.175–6.400 s CPU, 671–680 → 846–869 MiB family peak. Small tasks remain about 4.3 s; warm selected reads take 33 ms. No universal speed/memory gain or actual model usage is claimed. [Methods, complete measurements and reproduction](https://github.com/morluto/rea/blob/perf/5499-go-metadata-views-20261020/experiments/analysis-efficiency/go-build-views.md). - [x] Observations, derivations and inferences remain distinguishable. - [x] Artifact identity, provenance and failed attempts are preserved. - [x] Missing metadata, partial parsing and runtime dependency completeness remain explicit. ## Validation performed Current head is rebased onto `main@cf5194f68`, retaining its new IL2CPP contract and the Go additions. All 258 focused tests in 12 files pass, including the 32 IL2CPP view tests and 13 CLI path tests. Full `npm run check`, `npm run docs:check` plus five README tests, and a rebuilt-runtime real SDK selected query/full export pass. The original complete parent identity and all 165 dependencies remain exact. Duplication passes at 3.49018% (baseline 3.49743%, gate 3.5%). The frozen timing cohorts below are not timings of this rebase. After rebasing onto `main@c374c2fe7`: 10 files / 202 focused tests pass; `npm run check` and `npm run docs:check` pass (including five README consistency tests). A real rebuilt-runtime SDK selected query and complete export pass again with the original parent identity and all 165 dependencies. Eight frozen matched SDK tasks plus a warm diagnostic, six actual-client task variants, and real CLI complete/summary/bundle/file-view verification also pass. All qualified payload exits and owned cleanup are verified, with zero observed OOM events. Two qualification attempts stopped at the host-memory guard, including a latest-main build; owned scopes were verified inactive and unchanged-guard retries passed after headroom recovered. Final whole-scope counters are unavailable. Review qualification: the shared benchmark finalizer now cleans owned resources before optional transcript publication. All 11 receipt-failure tests pass; a real selected SDK task/export passes, and an injected transcript-write failure exits 1 with a recorded failure and verified process-family cleanup. Architecture review: Go domain now owns the existing report-to-subject architecture mapping. Production Evidence and every Go view share it, rejecting inconsistent inline/file parents, including conflicts between known and unknown architectures. All 213 focused tests and full `npm run check` pass after this correction. The initial path-message/duplication failures disappeared on new main. Current CI shard 4 has four `filteredFailureOutput.test.ts` assertions expecting the old generic remediation text; all four reproduce on untouched `main@cf5194f68` (`npm run test:focused -- tests/boundary/cli/filteredFailureOutput.test.ts`: 4 failed, 1 passed). Actual output uses the contract-aware recovery guidance added by main. The Go PR does not change that workflow. The control run used the pinned toolchain, bounded resources, verified cleanup and zero observed OOM events. Runtime/platform checks, including the Windows lane, passed. Matrix fail-fast cancelled shards 3 and 5; a direct shard-3 rerun was cancelled again because the original shard-4 failure remains. Their complete CI coverage is unverified; changed Go/Rust/CLI/MCP paths are covered by the 258 focused local tests and real SDK/CLI tasks. The owner explicitly authorized bypass for a verified PR blocked by unrelated CI; the bypass is for this reproduced baseline failure and its resulting aggregates/cancellations. ## Compatibility, safety, and release No default reply or complete Evidence identity change. Real verification covers Linux ELF; PE/Mach-O use authored fixtures. Metadata observes embedded claims and does not authenticate a compiler or establish complete runtime dependencies. Actual model reasoning, backend schema acceptance, rendering and billing are unmeasured. REA never executes an input. The installed Go tool is also invoked as the trusted native verifier. ## Author readiness Human owner: @gensokyo-dev The owner explicitly dictated these declarations and initial readiness in the working session; the assistant entered them under that instruction. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved; no additional maintainer scope decision is required. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked. - [x] Tests cover changed behavior and meaningful failure paths. - [x] Owning docs, contracts and generated metadata are updated where needed. - [x] Representative CLI/MCP output and measured limits are documented. - [x] The final diff has been checked for secrets, unrelated cleanup and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:44:17 +00:00