morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #6938] [Feature] Query selected BEAM table facts without delivering the full report #4243

New issue

Closed

opened 2026-10-31 00:48:02 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:48:02 +00:00

Owner

Originally created by @gensokyo-dev on GitHub (Oct 24, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6938

Before submitting

Searched open/closed issues and PRs; #4833 supplies the current reader, #6503/#6658 cover distinct parser/cancellation bugs. No matching selected-view proposal found.
Reviewed contribution scope; this proposal queries existing static facts and adds no target execution, injection or mutation authority.
Uses public distribution samples and contains no credentials/private samples.

Area

MCP or agent integration; Evidence, artifacts, or reconstruction; CLI.

Current limitation

At e2463372eae6dcd0ec558d1e4668cfc3814c04a0, answering “which exports named module/exprs exist, with their arities, labels and atom indexes?” requires the complete inspect_erlang_beam result. The named contract has no detail/view selection; retained-view dispatch has no BEAM parent projector (source-derived). Actual production stdio SDK tasks confirm that unrelated atom/import/lambda inventories dominate the answer.

Observed on Debian's unmodified Erlang/OTP27 package, with exact SDK tool definitions/default validation and complete source export:

Artifact	Needed exports	MCP text-reply estimate	Reply bytes	Whole-task estimate	Wall / family CPU / peak
erl_lint.beam	4	20,870 tokens	146,965	76,897 tokens	6.671s / 6.214s / 663.7MiB
erl_parse.beam	3	16,461 tokens	110,419	72,489 tokens	6.568s / 6.225s / 637.9MiB
pg2.beam (small)	2	1,707 tokens	11,057	57,725 tokens	6.369s / 5.959s / 642.7MiB

Token estimates use tiktoken0.14.0/cl100k_base: question, server instructions, all221 input-tool names/descriptions/schemas once, all arguments and text replies including export. Raw catalog including output schemas is829,075 estimated tokens, reported separately. Actual model usage/inference quality is unmeasured. One fresh baseline task per input, not repeated before/after performance evidence. Node compile cache disabled; OS caches uncontrolled.

Use cases

Read a few exported signatures or imports from a real module; follow their atom indexes without ingesting all other tables. erl_lint has915 atoms,119 imports and112 lambdas for just4 selected exports. Same-name exports with different arities must all survive.

Desired outcome and success criteria

Add opt-in initial summary/selected delivery and retained/offline beam table views, keeping complete output as default and complete Evidence accessible.
Select atoms/exports/imports/lambdas/chunks by exact applicable names or original row indexes, choose fields and page matches in source order. Keep original atom indexes and export arities; identify rows with parent+collection+source_index.
Preserve missing tables (null), decoded empty tables, unresolved atom names, source byte ranges/digests, coverage and limitations. Page coverage must not imply that missing structural tables establish runtime absence.
Share the domain projector/application workflow across CLI/MCP. Reuse existing immutable snapshot ownership and CLI delivery parsing; avoid another parser or long-lived path cache.
Validate default compatibility, exact converted JSON Schema clients, duplicate-name alternatives, empty selectors/fields, malformed data, source deletion/offline queries and delivery refusals. Repeat completed full/selected tasks on both large real samples plus the small case; report whole-task token/wall/family CPU/peak memory, including catalog changes and export. Benefits remain unmeasured until then.

Expected interaction with targets

Read existing local files; optionally publish complete Evidence through the existing caller-selected output path. No selected BEAM bytecode execution. Sample acquisition is separate preparation over HTTPS.

Target and scope details

Linux x86-64; existing bundled static reader. Public Debian erlang-base_27.3.4.1+dfsg-1+deb13u3_amd64.deb SHA256 4ebf03a400bc44420df4f1b507718413fcb42ed64b979bca05778722e777da13. Sample SHA256: erl_lint 716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55; erl_parse 729cce2b1688a1613d1ee71f5d0d99a9430d163d08fbe1aa5148dbbe6494f458; pg2 f01d6f2da046796ec572657ca10cb73c3193d42663315d5446f29aeb5821b5ef.

Verification available so far

Observed results. Three real production SDK tasks passed independent Python struct decoding of every table, chunk range/hash and Code header, selected answers and complete Evidence export. This oracle deliberately covers the fixed OTP27 sample encoding, not all historical/compact formats. No Erlang installation/emulator or target execution was used. One-CPU quota,3GiB early stop/4GiB hard cap, no swap; all payload exits/cleanup verified without a stop/OOM. Final whole-scope counters unavailable. Logs/oracles/identity receipts are retained locally for the follow-up benchmark; new-view behavior is not implemented or claimed verified.

Alternatives considered

Client-side filtering still delivers the entire report. Lower parser budgets refuse inputs and change examined coverage; they are not query limits. Raw external chunk inspection loses the shared Evidence/client workflow. Existing PYC/native-export selected-view patterns are relevant prior art.

Possible direction

Reuse named analysis detail contracts, existing retention/publication ownership, shared schema-specific immutable-result reader and a typed domain projector over the owning BEAM schema. Selection changes delivery, not parsing scope or target authority.

References

Reader contract, domain facts, official beam_lib semantics, public Debian package. #6928/#6907 demonstrate existing retained-view patterns, not BEAM performance guarantees.

Originally created by @gensokyo-dev on GitHub (Oct 24, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6938 ### Before submitting - [x] Searched open/closed issues and PRs; #4833 supplies the current reader, #6503/#6658 cover distinct parser/cancellation bugs. No matching selected-view proposal found. - [x] Reviewed contribution scope; this proposal queries existing static facts and adds no target execution, injection or mutation authority. - [x] Uses public distribution samples and contains no credentials/private samples. ### Area MCP or agent integration; Evidence, artifacts, or reconstruction; CLI. ### Current limitation At `e2463372eae6dcd0ec558d1e4668cfc3814c04a0`, answering “which exports named module/exprs exist, with their arities, labels and atom indexes?” requires the complete `inspect_erlang_beam` result. The named contract has no detail/view selection; retained-view dispatch has no BEAM parent projector (source-derived). Actual production stdio SDK tasks confirm that unrelated atom/import/lambda inventories dominate the answer. Observed on Debian's unmodified Erlang/OTP27 package, with exact SDK tool definitions/default validation and complete source export: | Artifact | Needed exports | MCP text-reply estimate | Reply bytes | Whole-task estimate | Wall / family CPU / peak | |---|---:|---:|---:|---:|---:| | erl_lint.beam | 4 | 20,870 tokens | 146,965 | 76,897 tokens | 6.671s / 6.214s / 663.7MiB | | erl_parse.beam | 3 | 16,461 tokens | 110,419 | 72,489 tokens | 6.568s / 6.225s / 637.9MiB | | pg2.beam (small) | 2 | 1,707 tokens | 11,057 | 57,725 tokens | 6.369s / 5.959s / 642.7MiB | Token estimates use tiktoken0.14.0/cl100k_base: question, server instructions, all221 input-tool names/descriptions/schemas once, all arguments and text replies including export. Raw catalog including output schemas is829,075 estimated tokens, reported separately. Actual model usage/inference quality is unmeasured. One fresh baseline task per input, not repeated before/after performance evidence. Node compile cache disabled; OS caches uncontrolled. ### Use cases Read a few exported signatures or imports from a real module; follow their atom indexes without ingesting all other tables. erl_lint has915 atoms,119 imports and112 lambdas for just4 selected exports. Same-name exports with different arities must all survive. ### Desired outcome and success criteria - Add opt-in initial summary/selected delivery and retained/offline `beam` table views, keeping complete output as default and complete Evidence accessible. - Select atoms/exports/imports/lambdas/chunks by exact applicable names or original row indexes, choose fields and page matches in source order. Keep original atom indexes and export arities; identify rows with parent+collection+source_index. - Preserve missing tables (`null`), decoded empty tables, unresolved atom names, source byte ranges/digests, coverage and limitations. Page coverage must not imply that missing structural tables establish runtime absence. - Share the domain projector/application workflow across CLI/MCP. Reuse existing immutable snapshot ownership and CLI delivery parsing; avoid another parser or long-lived path cache. - Validate default compatibility, exact converted JSON Schema clients, duplicate-name alternatives, empty selectors/fields, malformed data, source deletion/offline queries and delivery refusals. Repeat completed full/selected tasks on both large real samples plus the small case; report whole-task token/wall/family CPU/peak memory, including catalog changes and export. Benefits remain unmeasured until then. ### Expected interaction with targets Read existing local files; optionally publish complete Evidence through the existing caller-selected output path. No selected BEAM bytecode execution. Sample acquisition is separate preparation over HTTPS. ### Target and scope details Linux x86-64; existing bundled static reader. Public Debian `erlang-base_27.3.4.1+dfsg-1+deb13u3_amd64.deb` SHA256 `4ebf03a400bc44420df4f1b507718413fcb42ed64b979bca05778722e777da13`. Sample SHA256: erl_lint `716d78394b7c3da983e36aac7aadd6673e5041677ba7de20413cc2704d958d55`; erl_parse `729cce2b1688a1613d1ee71f5d0d99a9430d163d08fbe1aa5148dbbe6494f458`; pg2 `f01d6f2da046796ec572657ca10cb73c3193d42663315d5446f29aeb5821b5ef`. ### Verification available so far Observed results. Three real production SDK tasks passed independent Python struct decoding of every table, chunk range/hash and Code header, selected answers and complete Evidence export. This oracle deliberately covers the fixed OTP27 sample encoding, not all historical/compact formats. No Erlang installation/emulator or target execution was used. One-CPU quota,3GiB early stop/4GiB hard cap, no swap; all payload exits/cleanup verified without a stop/OOM. Final whole-scope counters unavailable. Logs/oracles/identity receipts are retained locally for the follow-up benchmark; new-view behavior is not implemented or claimed verified. ### Alternatives considered Client-side filtering still delivers the entire report. Lower parser budgets refuse inputs and change examined coverage; they are not query limits. Raw external chunk inspection loses the shared Evidence/client workflow. Existing PYC/native-export selected-view patterns are relevant prior art. ### Possible direction Reuse named analysis detail contracts, existing retention/publication ownership, shared schema-specific immutable-result reader and a typed domain projector over the owning BEAM schema. Selection changes delivery, not parsing scope or target authority. ### References [Reader contract](https://github.com/morluto/rea/blob/e2463372eae6dcd0ec558d1e4668cfc3814c04a0/src/contracts/erlang/beamToolContracts.ts), [domain facts](https://github.com/morluto/rea/blob/e2463372eae6dcd0ec558d1e4668cfc3814c04a0/src/domain/erlang/beam.ts), [official beam_lib semantics](https://www.erlang.org/doc/apps/stdlib/beam_lib.html), [public Debian package](https://deb.debian.org/debian/pool/main/e/erlang/erlang-base_27.3.4.1%2Bdfsg-1%2Bdeb13u3_amd64.deb). #6928/#6907 demonstrate existing retained-view patterns, not BEAM performance guarantees.

bsakaguchi

2026-10-31 00:48:02 +00:00

closed this issue
added the
triage

enhancement
labels

bsakaguchi commented

2026-10-31 00:48:03 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 24, 2026):

Strong +1 for selected-view delivery. The numbers speak for themselves: erl_lint.beam drops from 76.9k to 20.9k estimated tokens, and pg2 from 57.7k to 1.7k — most SDK calls only need a handful of exports/atoms and the full catalog is pure noise. A few design points I'd like to see kept: complete output as default so the public contract stays stable; preserve original atom indexes and export arities so same-name/different-arity exports all survive; and when paging coverage is reported, a missing structural table must not imply runtime absence. Reusing the existing immutable snapshot ownership and CLI delivery parsing across CLI/MCP is the right call — another parser or long-lived path cache would just add maintenance surface. Eager to test once the validation matrix (malformed data, source deletion/offline queries, empty selectors) lands.

@XinwangTao commented on GitHub (Oct 24, 2026): Strong +1 for selected-view delivery. The numbers speak for themselves: erl_lint.beam drops from 76.9k to 20.9k estimated tokens, and pg2 from 57.7k to 1.7k — most SDK calls only need a handful of exports/atoms and the full catalog is pure noise. A few design points I'd like to see kept: complete output as default so the public contract stays stable; preserve original atom indexes and export arities so same-name/different-arity exports all survive; and when paging coverage is reported, a missing structural table must not imply runtime absence. Reusing the existing immutable snapshot ownership and CLI delivery parsing across CLI/MCP is the right call — another parser or long-lived path cache would just add maintenance surface. Eager to test once the validation matrix (malformed data, source deletion/offline queries, empty selectors) lands.

bsakaguchi referenced this issue

2026-10-31 01:12:11 +00:00

[PR #4243] [MERGED] fix(javascript): preserve incoming callback target uncertainty #6142

bsakaguchi referenced this issue

2026-10-31 01:12:34 +00:00

[PR #4347] [CLOSED] fix(javascript): escape repeated prefixes in front of copied-slot selections #6192

bsakaguchi referenced this issue

2026-10-31 01:12:34 +00:00

[PR #4349] [CLOSED] fix(javascript): close full-read alias cycles at the binding in call resolution #6193

bsakaguchi referenced this issue

2026-10-31 01:12:39 +00:00

[PR #4363] [CLOSED] fix(javascript): grow a binding's returned-reference projection once per chain #6205