morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4987] [Feature] Let analyze-javascript-application select sources by folder or glob #3083

New issue

Open

opened 2026-10-31 00:36:16 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:36:16 +00:00

Owner

Originally created by @KJLavender on GitHub (Oct 18, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4987

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

CLI

Current limitation

I was trying REA on a fairly big Electron app on Windows. Its app.asar has about 4.7k JS files, and the main-process bundle alone is 9.5 MB.

Running it on the whole asar didn't work. After about 8 minutes it stopped with resource_constraint ("The parent's result capacity was exhausted ... remaining sources were not analyzed"), and the advice was to retry with more memory or a smaller artifact. #3538 already covers the memory half of that advice. My problem is the other half: there isn't a practical way to make the input smaller.

All I really needed was the Electron side, out/main/** and out/preload/**, which is 210 files. --source-path only accepts exact file paths. A folder (out/preload), a trailing slash, or a glob (out/preload/*, out/preload/**) all fail with a generic invalid_request ("Analysis input is invalid"), while out/preload/index.js works fine. So I generated the full list instead. That's about 13.5k characters of arguments, and through npx on Windows it never started: cmd.exe refused it with "The command line is too long". (My Windows is in Chinese, so what I actually saw was 命令列太長。)

What finally worked was extracting the asar with @electron/asar, copying out/main, out/preload and package.json into a new folder, and analyzing that folder. That's fine for poking around, but now the Evidence subject is my temp folder and not the shipped app.asar and its digest. As I understand the docs, keeping the original inventory and identity intact is exactly what source_paths is for.

Use cases

Electron apps in general. The IPC handlers live in main and preload, which is usually a small slice of the asar. In my case the whole-asar run ran out of budget somewhere in the renderer chunks under out/web/assets/.
Agents over MCP hit a softer version of this. They don't have the command-line limit, but they have to inventory the app and then send back hundreds of exact paths (13.5k characters in my case) just to say "only these two folders".

Desired outcome and success criteria

A way to say "everything under these folders" or "files matching this pattern", in both the CLI and MCP. Something like:

rea analyze-javascript-application app.asar --source-glob "out/main/**" --source-glob "out/preload/**"

A list file (--source-paths-from list.txt) would also solve the Windows part, even without patterns.

It's a success if the same 210-file selection can run directly against the original app.asar, the Evidence keeps the asar identity, and the unselected files are reported as unknowns the same way --source-path reports them now.

Expected interaction with targets

Read existing files or captured data

Verification available so far

I have a prototype or observed results

Verification details (optional)

rea-agents 6.5.0 (latest on npm right now), Node 24.20.0, Windows 11 Pro x64, 32 GB RAM:

Run	Time	Result
Whole `app.asar`	~8 min	`resource_constraint`, stopped at file 3,831 of 4,724. It still wrote a 338 MB JSON error envelope
210 × `--source-path` through npx	< 2 s	cmd.exe: command line too long
Copied folder (211 files), default heap	~1.5 min	exit 0, but `main_handlers: 0` and `paired_renderer_transmissions: 0`
Same folder with `--max-heap-mb 6144`	~2 min	`main_handlers: 791`, `paired_renderer_transmissions: 757`

Side note on the default-heap row, since it confused me for a while: the run exits 0 and the summary says main_handlers: 0. The only explanation is in limitations: "Owned JavaScript worker exhausted its heap while analyzing out/main/index.js during static". That sentence looks cut off, and it doesn't mention --max-heap-mb. I only found the flag by reading --help. This seems close to #4601 and #3671, so I'm not asking for it here. Just mentioning it in case a hint in that message is cheap to add.

Alternatives considered

Repeating --source-path: runs into the Windows command-line limit, and you have to build the list yourself.
Extracting the asar and copying folders: works, but loses the asar identity, as above.

Possible direction

Accept folder prefixes or glob patterns (in source_paths or a separate field) and expand them against the inventory REA already builds, so the final selection is still exact and the expanded list can be echoed back in the Evidence. For the CLI, reading the list from a file or stdin would help even before patterns exist.

Originally created by @KJLavender on GitHub (Oct 18, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4987 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area CLI ### Current limitation I was trying REA on a fairly big Electron app on Windows. Its `app.asar` has about 4.7k JS files, and the main-process bundle alone is 9.5 MB. Running it on the whole asar didn't work. After about 8 minutes it stopped with `resource_constraint` ("The parent's result capacity was exhausted ... remaining sources were not analyzed"), and the advice was to retry with more memory or a smaller artifact. #3538 already covers the memory half of that advice. My problem is the other half: there isn't a practical way to make the input smaller. All I really needed was the Electron side, `out/main/**` and `out/preload/**`, which is 210 files. `--source-path` only accepts exact file paths. A folder (`out/preload`), a trailing slash, or a glob (`out/preload/*`, `out/preload/**`) all fail with a generic `invalid_request` ("Analysis input is invalid"), while `out/preload/index.js` works fine. So I generated the full list instead. That's about 13.5k characters of arguments, and through `npx` on Windows it never started: cmd.exe refused it with "The command line is too long". (My Windows is in Chinese, so what I actually saw was 命令列太長。) What finally worked was extracting the asar with `@electron/asar`, copying `out/main`, `out/preload` and `package.json` into a new folder, and analyzing that folder. That's fine for poking around, but now the Evidence subject is my temp folder and not the shipped `app.asar` and its digest. As I understand the docs, keeping the original inventory and identity intact is exactly what `source_paths` is for. ### Use cases - Electron apps in general. The IPC handlers live in main and preload, which is usually a small slice of the asar. In my case the whole-asar run ran out of budget somewhere in the renderer chunks under `out/web/assets/`. - Agents over MCP hit a softer version of this. They don't have the command-line limit, but they have to inventory the app and then send back hundreds of exact paths (13.5k characters in my case) just to say "only these two folders". ### Desired outcome and success criteria A way to say "everything under these folders" or "files matching this pattern", in both the CLI and MCP. Something like: ``` rea analyze-javascript-application app.asar --source-glob "out/main/**" --source-glob "out/preload/**" ``` A list file (`--source-paths-from list.txt`) would also solve the Windows part, even without patterns. It's a success if the same 210-file selection can run directly against the original `app.asar`, the Evidence keeps the asar identity, and the unselected files are reported as unknowns the same way `--source-path` reports them now. ### Expected interaction with targets Read existing files or captured data ### Verification available so far I have a prototype or observed results ### Verification details (optional) rea-agents 6.5.0 (latest on npm right now), Node 24.20.0, Windows 11 Pro x64, 32 GB RAM: | Run | Time | Result | |---|---|---| | Whole `app.asar` | ~8 min | `resource_constraint`, stopped at file 3,831 of 4,724. It still wrote a 338 MB JSON error envelope | | 210 × `--source-path` through npx | < 2 s | cmd.exe: command line too long | | Copied folder (211 files), default heap | ~1.5 min | exit 0, but `main_handlers: 0` and `paired_renderer_transmissions: 0` | | Same folder with `--max-heap-mb 6144` | ~2 min | `main_handlers: 791`, `paired_renderer_transmissions: 757` | Side note on the default-heap row, since it confused me for a while: the run exits 0 and the summary says `main_handlers: 0`. The only explanation is in `limitations`: "Owned JavaScript worker exhausted its heap while analyzing out/main/index.js during static". That sentence looks cut off, and it doesn't mention `--max-heap-mb`. I only found the flag by reading `--help`. This seems close to #4601 and #3671, so I'm not asking for it here. Just mentioning it in case a hint in that message is cheap to add. ### Alternatives considered - Repeating `--source-path`: runs into the Windows command-line limit, and you have to build the list yourself. - Extracting the asar and copying folders: works, but loses the asar identity, as above. ### Possible direction Accept folder prefixes or glob patterns (in `source_paths` or a separate field) and expand them against the inventory REA already builds, so the final selection is still exact and the expanded list can be echoed back in the Evidence. For the CLI, reading the list from a file or stdin would help even before patterns exist.

bsakaguchi commented

2026-10-31 00:36:17 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 18, 2026):

This is a well-scoped request — the whole-asar run dying on renderer chunks when you only need main/preload is exactly the case a selectable source set should cover. +1 on expanding patterns against the inventory REA already builds rather than the filesystem: for an app.asar target there is no filesystem layout to glob against (the asar is opaque to the CLI), so inventory-based expansion is the only approach that keeps the asar identity and lets the expanded list be echoed back in Evidence. Two small additions to consider: (1) land --source-paths-from <file> (or stdin) even before globs — it's the cheapest fix for the Windows command-line limit and also helps MCP clients that can't comfortably carry hundreds of paths in one call; (2) for the MCP surface the file-based option won't map, so the inventory-glob matters most there — an array field that accepts folder prefixes would already cover it. On the heap hint: yes, please reference --max-heap-mb in that truncated "Owned JavaScript worker exhausted its heap while analyzing out/main/index.js during static" message — as it stands it reads like a hard failure, and the exit-0 summary with main_handlers: 0 makes it look like an empty result rather than a partial one.

@XinwangTao commented on GitHub (Oct 18, 2026): This is a well-scoped request — the whole-asar run dying on renderer chunks when you only need main/preload is exactly the case a selectable source set should cover. +1 on expanding patterns against the inventory REA already builds rather than the filesystem: for an app.asar target there is no filesystem layout to glob against (the asar is opaque to the CLI), so inventory-based expansion is the only approach that keeps the asar identity and lets the expanded list be echoed back in Evidence. Two small additions to consider: (1) land `--source-paths-from <file>` (or stdin) even before globs — it's the cheapest fix for the Windows command-line limit and also helps MCP clients that can't comfortably carry hundreds of paths in one call; (2) for the MCP surface the file-based option won't map, so the inventory-glob matters most there — an array field that accepts folder prefixes would already cover it. On the heap hint: yes, please reference --max-heap-mb in that truncated "Owned JavaScript worker exhausted its heap while analyzing out/main/index.js during static" message — as it stands it reads like a hard failure, and the exit-0 summary with main_handlers: 0 makes it look like an empty result rather than a partial one.

bsakaguchi commented

2026-10-31 00:36:18 +00:00

Author

Owner

@s4h8a commented on GitHub (Oct 18, 2026):

Claiming the folder/glob part of this.

#5029 (@KJLavender) covers the complementary first step: taking a long list of
exact paths from a JSON array or file. This change covers the selector
semantics themselves, resolved against the complete artifact inventory (never
the filesystem, so opaque ASAR containers keep their identity):

an entry that exactly names an inventoried path selects it;
otherwise a folder prefix selects every inventoried file under its slash
boundary (trailing slash ignored);
otherwise a glob matches with * within one path segment, ** across zero
or more whole segments and ? for one character (case-sensitive,
code-point ordered).

First-request order with deduping; overlapping selectors keep first-request
priority; only unmatched selectors fail, with one invalid_request issue per
index. source_selection.selected_paths records the resolved exact list, and
only when a folder or glob resolved sources, so exact-only output stays
byte-identical. Draft PR coming shortly.

@s4h8a commented on GitHub (Oct 18, 2026): Claiming the folder/glob part of this. #5029 (@KJLavender) covers the complementary first step: taking a long list of exact paths from a JSON array or file. This change covers the selector semantics themselves, resolved against the complete artifact inventory (never the filesystem, so opaque ASAR containers keep their identity): - an entry that exactly names an inventoried path selects it; - otherwise a folder prefix selects every inventoried file under its slash boundary (trailing slash ignored); - otherwise a glob matches with `*` within one path segment, `**` across zero or more whole segments and `?` for one character (case-sensitive, code-point ordered). First-request order with deduping; overlapping selectors keep first-request priority; only unmatched selectors fail, with one `invalid_request` issue per index. `source_selection.selected_paths` records the resolved exact list, and only when a folder or glob resolved sources, so exact-only output stays byte-identical. Draft PR coming shortly.

bsakaguchi referenced this issue

2026-10-31 00:38:23 +00:00

[GH-ISSUE #5342] [Bug] Server-wide REA_GHIDRA_DISABLED_ANALYZERS naming a format-specific analyzer stops other formats from opening #3250

bsakaguchi referenced this issue

2026-10-31 00:39:31 +00:00

[GH-ISSUE #5523] [Feature] inspect_native_api: report whether a function is variadic #3360

bsakaguchi referenced this issue

2026-11-01 21:10:57 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue

2026-11-01 21:10:59 +00:00

[GH-ISSUE #5523] [Feature] inspect_native_api: report whether a function is variadic #3360