[PR #6928] [MERGED] feat(python): query selected pyc code-object facts #7453

Closed
opened 2026-11-01 21:47:58 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6928
Author: @gensokyo-dev
Created: 10/24/2026
Status: ✅ Merged
Merged: 10/24/2026
Merged by: @gensokyo-dev

Base: main ← Head: perf/6922-pyc-selected-views-20261023


📝 Commits (10+)

  • 97ea826 feat(python): query retained pyc code-object facts
  • 71d94e8 Merge commit '6b3eee1ccf' into perf/6922-pyc-selected-views-20261023
  • 6b1c6cd test(python): bind complete pyc selection benchmark tasks
  • 29ff089 test(python): include server instructions in token accounting
  • c172fd3 chore: sync pyc views with current main
  • 94f6c67 docs(perf): record pyc selected-facts task measurements
  • 062f94c test(python): keep oracle dependencies separate from selected targets
  • c9d2b4c docs(perf): record guarded oracle requalification
  • d3f9191 chore: integrate current main analysis-view contracts
  • 638fd9b test(python): narrow complete Evidence through its owning contract

📊 Changes

22 files changed (+3664 additions, -156 deletions)

View changed files

📝 docs/python-pyc.md (+54 -0)
➕ experiments/analysis-efficiency/python-pyc-costs.py (+114 -0)
➕ experiments/analysis-efficiency/python-pyc-oracle.py (+103 -0)
➕ experiments/analysis-efficiency/python-pyc-views.json (+2265 -0)
➕ experiments/analysis-efficiency/python-pyc-views.md (+59 -0)
➕ experiments/analysis-efficiency/python-pyc-views.mjs (+272 -0)
📝 src/cli/analysisEvidenceDelivery.ts (+51 -1)
📝 src/cli/javascriptRouteDelivery.ts (+2 -30)
📝 src/cli/pythonPycCommands.ts (+47 -26)
📝 src/contracts/analysisViewToolContracts.ts (+1 -1)
📝 src/contracts/python/pythonPycToolContracts.ts (+46 -4)
➕ src/contracts/pythonPycDeliveryContracts.test.ts (+58 -0)
📝 src/domain/analysisView/analysisView.ts (+27 -4)
➕ src/domain/analysisView/analysisViewParent.test.ts (+52 -0)
📝 src/domain/analysisView/analysisViewParent.ts (+32 -0)
📝 src/domain/analysisView/nativeExportsView.ts (+2 -31)
➕ src/domain/analysisView/pythonPycView.ts (+145 -0)
📝 src/server/createServer.ts (+1 -0)
📝 src/server/registerPythonPycTool.ts (+36 -3)
➕ tests/acceptance/applications/pythonPycViews.test.ts (+257 -0)

...and 2 more files

📄 Description

Summary

Fixes #6922. Read selected PYC code-object facts in the initial inspection, with filters, fields and paging over the same complete retained Evidence. Complete output remains the default.

Problem and expected behavior

Looking up three named functions previously delivered the entire report (194/307 objects for real argparse/typing). detail: "summary", view: {kind: "python-pyc", collection: "code_objects", names: [...], fields: [...]} now returns the matching facts immediately. Same-name alternatives retain original parent-scoped indexes; nested_code remains a code-constant relationship. CLI summary delivery saves the complete bundle for later offline views.

Change and scope

The domain projector owns selection and source/page coverage. Native-export and PYC views share a schema-specific immutable-result reader; CLI delivery JSON parsing is shared with existing JavaScript routes. Providers still parse/retain the complete report with existing budgets. Maintainer execution/injection/mutation scope decision: not applicable; this extends existing static inspection.

Contract and boundary impact

  • CLI/MCP: additive opt-in detail/view; complete delivery rejects a supplied view. Exact named contracts and converted draft-2020-12/draft-07 inputs are tested.
  • Evidence: parent IDs, source coverage, complete facts and export remain available; retained/offline queries work after deleting the source. Cache lifetime follows owner-sealed immutable facts, never paths/live state.
  • Compatibility/platform: existing CPython formats and default complete workflow retained; actual new-view qualification covers CPython 3.11.2 artifacts on Linux. Other real Python versions/platforms unverified here.
  • Generated docs/catalogs regenerated and checked; ignored generated outputs remain uncommitted. No installation/configuration change.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration. (Only caller-selected Evidence bundle publication in the existing export/delivery workflow.)
  • Contact network services.
  • Recover protected code or work around protections.

Evidence and regression coverage

Executed 18 alternating fresh production stdio SDK tasks against independent CPython graph/bytecode facts: argparse, typing and a small compiler-produced duplicate-name control. Both lanes answer the same question and export/validate the full report; every answer, parent ID and normalized-source digest agrees.

Whole-task estimated tokens (question + server instructions + all 221 input-tool descriptions/schemas once + all arguments/text replies including export): 104,415 → 57,300 for argparse; 128,858 → 57,316 for typing; small control 57,719 → 57,332. tiktoken 0.14.0/cl100k_base, scripted client, actual model usage unmeasured. No stable general elapsed/CPU/memory improvement established. Fixed revisions, ranges, complete raw-catalog costs and resource receipts are in experiments/analysis-efficiency/python-pyc-views.md and .json.

Validation performed

  • Integrated main 09c2a2d24; the later CI composite used main d81d6b670. Current head 638fd9be9 changes only two test consumers relative to CI-tested d3f919146, narrowing the complete Evidence branch through its owning output contract instead of validating it twice.
  • Shared view/reader, PYC/JavaScript delivery contracts and Evidence file-source tests: 108 passed. Current public PYC CLI/MCP, PYC views and native-export views: 27 passed using the prepared exact-test runner with one worker.
  • Serialized typecheck/lint/format/knip/metadata plus compile dependency and architecture guards passed. Generated documentation checked; 9 README tests passed.
  • Six additional production stdio/full-export tasks compare clean pinned 09c2a2d24 and 638fd9be9 against the corrected independent oracle. All facts/answers, parent IDs and source digests agree. Whole-task token estimates: argparse 104,533 → 57,418; typing 128,978 → 57,436; control 57,837 → 57,450. One pair per target qualifies integration, not a repeated speed/memory claim; the original 18-task table remains fixed to its recorded revisions.
  • Current head CI 40052834237 passed, testing composite 3d36c4d2 against main 9df94a62: 17,169 source tests passed, 13 skipped; all 8 public/built test shards passed, plus static/build/docs/reverse-tools quality. Earlier baseline failures and controls remain in the qualification comment. Codex review quota prevented a completed automated review; no approval is claimed.
  • One aggregate CPU, 3 GiB early stop/4 GiB hard cap, no swap. Completed measured tasks and final affected checks verify payload exit/cleanup without a resource stop/OOM; final whole-scope counters remain unavailable. Failed attempts remain recorded. Earlier public CLI timeouts were resolved by removing duplicate contract parsing in the test consumer; the current 27-test run passed.

Compatibility, safety, and release

Additive CLI/MCP selection, shared static provider/Evidence semantics. Real Hopper/Ghidra/browser verification is not applicable. Self-review found that the initial oracle CLI imported the system argparse target as an argument-parser dependency (outside measured tasks). It now uses getopt, guards argparse/typing module loading, and regenerates byte-for-byte equivalent oracle JSON; a fresh compiled control and all three real SDK tasks pass after the correction. The fixed-version table remains tied to its recorded revisions; production requires no Python installation. Whole source parsing/retention costs remain.

Author readiness

The requesting owner explicitly authorized ready PR creation and merge without repeated human declarations in this session. The human-review declarations are left unchecked.

Human owner: not recorded in the PR.

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6928 **Author:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Created:** 10/24/2026 **Status:** ✅ Merged **Merged:** 10/24/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `perf/6922-pyc-selected-views-20261023` --- ### 📝 Commits (10+) - [`97ea826`](https://github.com/morluto/rea/commit/97ea8265f0f5c0ec8ddc8c3c175dbcda5dc179da) feat(python): query retained pyc code-object facts - [`71d94e8`](https://github.com/morluto/rea/commit/71d94e8ecf4d2f19f1b1cc5afa82c31a55b1b4eb) Merge commit '6b3eee1ccf4c51f23bef7a0007c7b8e892f662db' into perf/6922-pyc-selected-views-20261023 - [`6b1c6cd`](https://github.com/morluto/rea/commit/6b1c6cd2e74551131a5d2c2ed64d198bdcd54ee5) test(python): bind complete pyc selection benchmark tasks - [`29ff089`](https://github.com/morluto/rea/commit/29ff0893e3a5a9ea9bc52f9f85a653388f6bc48a) test(python): include server instructions in token accounting - [`c172fd3`](https://github.com/morluto/rea/commit/c172fd3305cbb419c6466f635be804b47af79d14) chore: sync pyc views with current main - [`94f6c67`](https://github.com/morluto/rea/commit/94f6c679120bfe8530cfc81356276415debbb479) docs(perf): record pyc selected-facts task measurements - [`062f94c`](https://github.com/morluto/rea/commit/062f94ce36cbc269fdae92101cfd7c10e81f35c1) test(python): keep oracle dependencies separate from selected targets - [`c9d2b4c`](https://github.com/morluto/rea/commit/c9d2b4c19180831870957411b78e6180f86d567d) docs(perf): record guarded oracle requalification - [`d3f9191`](https://github.com/morluto/rea/commit/d3f9191463de2aaa85952518b309040a1358879c) chore: integrate current main analysis-view contracts - [`638fd9b`](https://github.com/morluto/rea/commit/638fd9be921095ae948031206328836fd5da1c32) test(python): narrow complete Evidence through its owning contract ### 📊 Changes **22 files changed** (+3664 additions, -156 deletions) <details> <summary>View changed files</summary> 📝 `docs/python-pyc.md` (+54 -0) ➕ `experiments/analysis-efficiency/python-pyc-costs.py` (+114 -0) ➕ `experiments/analysis-efficiency/python-pyc-oracle.py` (+103 -0) ➕ `experiments/analysis-efficiency/python-pyc-views.json` (+2265 -0) ➕ `experiments/analysis-efficiency/python-pyc-views.md` (+59 -0) ➕ `experiments/analysis-efficiency/python-pyc-views.mjs` (+272 -0) 📝 `src/cli/analysisEvidenceDelivery.ts` (+51 -1) 📝 `src/cli/javascriptRouteDelivery.ts` (+2 -30) 📝 `src/cli/pythonPycCommands.ts` (+47 -26) 📝 `src/contracts/analysisViewToolContracts.ts` (+1 -1) 📝 `src/contracts/python/pythonPycToolContracts.ts` (+46 -4) ➕ `src/contracts/pythonPycDeliveryContracts.test.ts` (+58 -0) 📝 `src/domain/analysisView/analysisView.ts` (+27 -4) ➕ `src/domain/analysisView/analysisViewParent.test.ts` (+52 -0) 📝 `src/domain/analysisView/analysisViewParent.ts` (+32 -0) 📝 `src/domain/analysisView/nativeExportsView.ts` (+2 -31) ➕ `src/domain/analysisView/pythonPycView.ts` (+145 -0) 📝 `src/server/createServer.ts` (+1 -0) 📝 `src/server/registerPythonPycTool.ts` (+36 -3) ➕ `tests/acceptance/applications/pythonPycViews.test.ts` (+257 -0) _...and 2 more files_ </details> ### 📄 Description ## Summary Fixes #6922. Read selected PYC code-object facts in the initial inspection, with filters, fields and paging over the same complete retained Evidence. Complete output remains the default. ## Problem and expected behavior Looking up three named functions previously delivered the entire report (194/307 objects for real argparse/typing). `detail: "summary", view: {kind: "python-pyc", collection: "code_objects", names: [...], fields: [...]}` now returns the matching facts immediately. Same-name alternatives retain original parent-scoped indexes; `nested_code` remains a code-constant relationship. CLI summary delivery saves the complete bundle for later offline views. ## Change and scope The domain projector owns selection and source/page coverage. Native-export and PYC views share a schema-specific immutable-result reader; CLI delivery JSON parsing is shared with existing JavaScript routes. Providers still parse/retain the complete report with existing budgets. Maintainer execution/injection/mutation scope decision: not applicable; this extends existing static inspection. ## Contract and boundary impact - CLI/MCP: additive opt-in detail/view; complete delivery rejects a supplied view. Exact named contracts and converted draft-2020-12/draft-07 inputs are tested. - Evidence: parent IDs, source coverage, complete facts and export remain available; retained/offline queries work after deleting the source. Cache lifetime follows owner-sealed immutable facts, never paths/live state. - Compatibility/platform: existing CPython formats and default complete workflow retained; actual new-view qualification covers CPython 3.11.2 artifacts on Linux. Other real Python versions/platforms unverified here. - Generated docs/catalogs regenerated and checked; ignored generated outputs remain uncommitted. No installation/configuration change. Expected effects: - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [x] Change process memory, registers, files, or configuration. (Only caller-selected Evidence bundle publication in the existing export/delivery workflow.) - [ ] Contact network services. - [ ] Recover protected code or work around protections. ## Evidence and regression coverage Executed 18 alternating fresh production stdio SDK tasks against independent CPython graph/bytecode facts: argparse, typing and a small compiler-produced duplicate-name control. Both lanes answer the same question and export/validate the full report; every answer, parent ID and normalized-source digest agrees. Whole-task estimated tokens (question + server instructions + all 221 input-tool descriptions/schemas once + all arguments/text replies including export): **104,415 → 57,300** for argparse; **128,858 → 57,316** for typing; small control **57,719 → 57,332**. `tiktoken 0.14.0/cl100k_base`, scripted client, actual model usage unmeasured. No stable general elapsed/CPU/memory improvement established. Fixed revisions, ranges, complete raw-catalog costs and resource receipts are in `experiments/analysis-efficiency/python-pyc-views.md` and `.json`. ## Validation performed - Integrated main `09c2a2d24`; the later CI composite used main `d81d6b670`. Current head `638fd9be9` changes only two test consumers relative to CI-tested `d3f919146`, narrowing the complete Evidence branch through its owning output contract instead of validating it twice. - Shared view/reader, PYC/JavaScript delivery contracts and Evidence file-source tests: **108 passed**. Current public PYC CLI/MCP, PYC views and native-export views: **27 passed** using the prepared exact-test runner with one worker. - Serialized typecheck/lint/format/knip/metadata plus compile dependency and architecture guards passed. Generated documentation checked; **9 README tests passed**. - Six additional production stdio/full-export tasks compare clean pinned `09c2a2d24` and `638fd9be9` against the corrected independent oracle. All facts/answers, parent IDs and source digests agree. Whole-task token estimates: argparse **104,533 → 57,418**; typing **128,978 → 57,436**; control **57,837 → 57,450**. One pair per target qualifies integration, not a repeated speed/memory claim; the original 18-task table remains fixed to its recorded revisions. - Current head CI [40052834237](https://github.com/morluto/rea/actions/runs/40052834237) **passed**, testing composite `3d36c4d2` against main `9df94a62`: **17,169 source tests passed, 13 skipped; all 8 public/built test shards passed**, plus static/build/docs/reverse-tools quality. Earlier baseline failures and controls remain in the qualification comment. Codex review quota prevented a completed automated review; no approval is claimed. - One aggregate CPU, 3 GiB early stop/4 GiB hard cap, no swap. Completed measured tasks and final affected checks verify payload exit/cleanup without a resource stop/OOM; final whole-scope counters remain unavailable. Failed attempts remain recorded. Earlier public CLI timeouts were resolved by removing duplicate contract parsing in the test consumer; the current 27-test run passed. ## Compatibility, safety, and release Additive CLI/MCP selection, shared static provider/Evidence semantics. Real Hopper/Ghidra/browser verification is not applicable. Self-review found that the initial oracle CLI imported the system argparse target as an argument-parser dependency (outside measured tasks). It now uses getopt, guards argparse/typing module loading, and regenerates byte-for-byte equivalent oracle JSON; a fresh compiled control and all three real SDK tasks pass after the correction. The fixed-version table remains tied to its recorded revisions; production requires no Python installation. Whole source parsing/retention costs remain. ## Author readiness The requesting owner explicitly authorized ready PR creation and merge without repeated human declarations in this session. The human-review declarations are left unchecked. Human owner: not recorded in the PR. - [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [ ] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-01 21:47:58 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7453
No description provided.