[PR #2801] [CLOSED] test(agent): grade Notes export facts and track fixture changes #5405

New issue

Closed

opened 2026-10-31 01:06:23 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:06:23 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2801
Author: @morluto
Created: 10/13/2026
Status: ❌ Closed

Base: main ← Head: codex/review-fixes-pr-2457

📝 Commits (10+)

ac82a18 test(agent): grade notes export facts against produced evidence
29711d1 fix(ci): classify Notes evaluation assets as code inputs
3cb84db chore: reconcile captured main with PR #2801
f5fe046 style(tests): format the Java class fixture signature
caadb20 chore: merge current main into PR #2801
432543d test(evaluation): provide complete fixture target
f3532d9 chore: merge main into PR #2801
d41f796 Merge commit '8fef2aa64e' into codex/ci-wave3-2801
7b33314 fix(ci): verify retained Windows directory ownership
360fc51 Merge branch 'main' into codex/review-fixes-pr-2457

📊 Changes

8 files changed (+501 additions, -16 deletions)

View changed files

📝 docs/testing.md (+3 -2)
📝 scripts/ci/plan.mjs (+3 -0)
📝 scripts/verify-agent-experience.mjs (+12 -0)
📝 scripts/verify-windows-native-inspection.mjs (+5 -2)
📝 src/evaluation/AgentEvaluationFixtures.ts (+144 -12)
📝 tests/boundary/filesystem/ciChangeScope.test.ts (+4 -0)
📝 tests/boundary/mcp/agentKnownAnswerMcp.test.ts (+328 -0)
📝 tests/evaluation/model-evals/knownAnswerEvaluation.test.ts (+2 -0)

📄 Description

Preserves the Notes export factual rubric from #2457: six closed claims are checked against the source-owned fixture and the delivered producer Evidence, including API, IPC, preload, module selection, and query boundaries.

The Notes example subtree is now also classified as a code/test input while retaining website checks. Changes to those fixture bytes therefore select the evaluation test lanes instead of passing as website-only changes.

Supersedes #2457. Original contribution by @optimization2026; commit ac82a1842491b59c72bb516ae14a534e1a7a874a remains unchanged in this branch's history.

Validation: source inspection, targeted formatting, and git diff --check. Tests, builds, lint, and CI reruns were skipped at the explicit user request. No new agent trial or real-provider qualification was performed.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2801 **Author:** [@morluto](https://github.com/morluto) **Created:** 10/13/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `codex/review-fixes-pr-2457` --- ### 📝 Commits (10+) - [`ac82a18`](https://github.com/morluto/rea/commit/ac82a1842491b59c72bb516ae14a534e1a7a874a) test(agent): grade notes export facts against produced evidence - [`29711d1`](https://github.com/morluto/rea/commit/29711d11fdc4e55f9bdf4c66f4ccb07d124b635d) fix(ci): classify Notes evaluation assets as code inputs - [`3cb84db`](https://github.com/morluto/rea/commit/3cb84db0026766d35e255e37b10c682d6e91e2cf) chore: reconcile captured main with PR #2801 - [`f5fe046`](https://github.com/morluto/rea/commit/f5fe0463d44804d5766e5d39bcb96fa32d29f0ea) style(tests): format the Java class fixture signature - [`caadb20`](https://github.com/morluto/rea/commit/caadb20563f7dae97287ce03e9eb878db252e373) chore: merge current main into PR #2801 - [`432543d`](https://github.com/morluto/rea/commit/432543def31bd219d86842f1121ad3297fdab94e) test(evaluation): provide complete fixture target - [`f3532d9`](https://github.com/morluto/rea/commit/f3532d9fb2128fd8c19aba2f7cc744c02eb7c9fb) chore: merge main into PR #2801 - [`d41f796`](https://github.com/morluto/rea/commit/d41f7963c81a8d1533af1899a5c031e85f6d2b71) Merge commit '8fef2aa64ec1ee742e8b70b825938d30a0ea7319' into codex/ci-wave3-2801 - [`7b33314`](https://github.com/morluto/rea/commit/7b33314701669f510997820bc263a99f73346bb0) fix(ci): verify retained Windows directory ownership - [`360fc51`](https://github.com/morluto/rea/commit/360fc5195edb705f0ea67297588a44497cf97d30) Merge branch 'main' into codex/review-fixes-pr-2457 ### 📊 Changes **8 files changed** (+501 additions, -16 deletions) <details> <summary>View changed files</summary> 📝 `docs/testing.md` (+3 -2) 📝 `scripts/ci/plan.mjs` (+3 -0) 📝 `scripts/verify-agent-experience.mjs` (+12 -0) 📝 `scripts/verify-windows-native-inspection.mjs` (+5 -2) 📝 `src/evaluation/AgentEvaluationFixtures.ts` (+144 -12) 📝 `tests/boundary/filesystem/ciChangeScope.test.ts` (+4 -0) 📝 `tests/boundary/mcp/agentKnownAnswerMcp.test.ts` (+328 -0) 📝 `tests/evaluation/model-evals/knownAnswerEvaluation.test.ts` (+2 -0) </details> ### 📄 Description Preserves the Notes export factual rubric from #2457: six closed claims are checked against the source-owned fixture and the delivered producer Evidence, including API, IPC, preload, module selection, and query boundaries. The Notes example subtree is now also classified as a code/test input while retaining website checks. Changes to those fixture bytes therefore select the evaluation test lanes instead of passing as website-only changes. Supersedes #2457. Original contribution by [@optimization2026](https://github.com/optimization2026); commit `ac82a1842491b59c72bb516ae14a534e1a7a874a` remains unchanged in this branch's history. Validation: source inspection, targeted formatting, and `git diff --check`. Tests, builds, lint, and CI reruns were skipped at the explicit user request. No new agent trial or real-provider qualification was performed. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:06:23 +00:00