[PR #2431] [MERGED] test(perf): observe owned counters at payload completion #5222

New issue

Closed

opened 2026-10-31 01:04:56 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:04:56 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2431
Author: @N0zoM1z0
Created: 10/13/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @N0zoM1z0

Base: main ← Head: test/resource-payload-counters-20261013

📝 Commits (2)

d1f2cfe test(perf): observe owned counters at payload completion
02bd831 test(perf): verify observer capacity and real workflow witnesses

📊 Changes

6 files changed (+384 additions, -4 deletions)

View changed files

📝 experiments/analysis-efficiency/README.md (+4 -0)
➕ experiments/analysis-efficiency/resource-counters.md (+103 -0)
📝 experiments/analysis-efficiency/resource-run.py (+59 -3)
➕ experiments/analysis-efficiency/resource_payload.py (+114 -0)
➕ experiments/analysis-efficiency/test_resource_payload.py (+103 -0)
📝 package.json (+1 -1)

📄 Description

Transient experiment cgroups can disappear before terminal counter reads, leaving earlier resource-qualified reports correctly unknown. Add optional --observe-payload-exit to capture kernel counters after the workload exits while its owned observer remains alive. Default process/receipt behavior, REA public APIs/providers and the existing full grader remain unchanged.

Validate the unit token, numeric fields, group identity, live observer membership, payload outcome and absence of additional processes before acknowledgement/verification. Record CPU, memory.peak and memory/task events with an explicit payload-exited-observer-alive phase. Unknown or inconsistent receipts remain unknown; lingering actors remain explicit. Preserve terminal whole-scope uncertainty. The opt-in observer requires at least two task slots and records supervisor/helper hashes before process acquisition; no CPU/memory/task budget is raised.

Validation:

Five new semantic tests plus six existing context/grader tests pass in the existing CI Python gate. Tests cover missing/malformed/wrong-owner/contradictory/boolean/negative/symlink receipts, unrelated metadata exclusion, nonzero events/status, lingering actors and capacity rejection before process acquisition.
Actual Linux systemd/cgroup probes preserve default shape, success/exit3, kernel peak for a bounded16MiB Node allocation, an unverified lingering child followed by scope cleanup, task-capacity stop125, and cancellation of a confirmed-live owned controller followed by scope cleanup. Unique-scope cleanup uses no bare-PID signaling.
Actual managed SDK default/observer pairs in both orders retain identical selected facts/settings and the same two calls. Actual SQLite WABT and NewPipe JADX/Java workflows also verify payload completion/owned group with only the observer remaining. The timing sample has an unexplained initial default outlier and is not a speed/overhead recommendation.
Actual Codex CLI0.161.0, requested gpt-5.6-terra high, passes the unchanged managed question, preserves source bytes and removes its profile. Three successful MCP calls include an extra repeated inspection. Witness: kernelpeak812,896,256 bytes, CPU8,566,029 usec and no limit events at payload exit; whole-scope final counters still unknown. Reported input61,173/cached48,128/output2,678 and125.798s are metering evidence, not a performance gain.
Full supported single-thread typecheck/dependency/lint pre-push passes under unchanged oneCPU/3GiB early-stop/4GiB hard/noSwap safeguards; normal pre-commit runs. Current checkpoint02bd83161fcbd84ff3b8ddeaf8b99149e374e046. All 39 exact-head verification checks passed; five release/publication checks were skipped. Final manual review completed before normal exact-head squash merge.

Starts from merged main4f4f6cd7b5d7a400b90772a138d7b9f942c2a556 in a new isolated worktree. Measured runtime remains frozen8bc160 and its original digest; previous observations are not relabeled. Curated findings: experiments/analysis-efficiency/resource-counters.md. Raw receipts/models/profiles stay private. Continues#2016; no release/deployment. Merged after verification at ca2d15b3ee on 2026-10-13 UTC; no release/deployment.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2431 **Author:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Created:** 10/13/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@N0zoM1z0](https://github.com/N0zoM1z0) **Base:** `main` ← **Head:** `test/resource-payload-counters-20261013` --- ### 📝 Commits (2) - [`d1f2cfe`](https://github.com/morluto/rea/commit/d1f2cfe76df28e6effe973123f2e19313f6d0d4b) test(perf): observe owned counters at payload completion - [`02bd831`](https://github.com/morluto/rea/commit/02bd83161fcbd84ff3b8ddeaf8b99149e374e046) test(perf): verify observer capacity and real workflow witnesses ### 📊 Changes **6 files changed** (+384 additions, -4 deletions) <details> <summary>View changed files</summary> 📝 `experiments/analysis-efficiency/README.md` (+4 -0) ➕ `experiments/analysis-efficiency/resource-counters.md` (+103 -0) 📝 `experiments/analysis-efficiency/resource-run.py` (+59 -3) ➕ `experiments/analysis-efficiency/resource_payload.py` (+114 -0) ➕ `experiments/analysis-efficiency/test_resource_payload.py` (+103 -0) 📝 `package.json` (+1 -1) </details> ### 📄 Description Transient experiment cgroups can disappear before terminal counter reads, leaving earlier resource-qualified reports correctly unknown. Add optional `--observe-payload-exit` to capture kernel counters after the workload exits while its owned observer remains alive. Default process/receipt behavior, REA public APIs/providers and the existing full grader remain unchanged. Validate the unit token, numeric fields, group identity, live observer membership, payload outcome and absence of additional processes before acknowledgement/verification. Record CPU, memory.peak and memory/task events with an explicit payload-exited-observer-alive phase. Unknown or inconsistent receipts remain unknown; lingering actors remain explicit. Preserve terminal whole-scope uncertainty. The opt-in observer requires at least two task slots and records supervisor/helper hashes before process acquisition; no CPU/memory/task budget is raised. Validation: - Five new semantic tests plus six existing context/grader tests pass in the existing CI Python gate. Tests cover missing/malformed/wrong-owner/contradictory/boolean/negative/symlink receipts, unrelated metadata exclusion, nonzero events/status, lingering actors and capacity rejection before process acquisition. - Actual Linux systemd/cgroup probes preserve default shape, success/exit3, kernel peak for a bounded16MiB Node allocation, an unverified lingering child followed by scope cleanup, task-capacity stop125, and cancellation of a confirmed-live owned controller followed by scope cleanup. Unique-scope cleanup uses no bare-PID signaling. - Actual managed SDK default/observer pairs in both orders retain identical selected facts/settings and the same two calls. Actual SQLite WABT and NewPipe JADX/Java workflows also verify payload completion/owned group with only the observer remaining. The timing sample has an unexplained initial default outlier and is not a speed/overhead recommendation. - Actual Codex CLI0.161.0, requested gpt-5.6-terra high, passes the unchanged managed question, preserves source bytes and removes its profile. Three successful MCP calls include an extra repeated inspection. Witness: kernelpeak812,896,256 bytes, CPU8,566,029 usec and no limit events at payload exit; whole-scope final counters still unknown. Reported input61,173/cached48,128/output2,678 and125.798s are metering evidence, not a performance gain. - Full supported single-thread typecheck/dependency/lint pre-push passes under unchanged oneCPU/3GiB early-stop/4GiB hard/noSwap safeguards; normal pre-commit runs. Current checkpoint02bd83161fcbd84ff3b8ddeaf8b99149e374e046. All 39 exact-head verification checks passed; five release/publication checks were skipped. Final manual review completed before normal exact-head squash merge. Starts from merged main4f4f6cd7b5d7a400b90772a138d7b9f942c2a556 in a new isolated worktree. Measured runtime remains frozen8bc160 and its original digest; previous observations are not relabeled. Curated findings: experiments/analysis-efficiency/resource-counters.md. Raw receipts/models/profiles stay private. Continues#2016; no release/deployment. Merged after verification at ca2d15b3ee965bd7ab27204cdf4b96b117d148c1 on 2026-10-13 UTC; no release/deployment. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:04:56 +00:00

closed this issue
added the
pull-request
label

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#5222

No description provided.

Rows
Columns