morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

~7.9 MB #983

New issue

Closed

opened 2026-10-15 18:30:06 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:30:06 +00:00

Owner

Originally created by @PixelPepper on GitHub (Oct 13, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/2722

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Process capture or runtime observation

Observed behavior

On Windows, rea inspect-wasm-artifact fails with provider_timeout (30000 ms) for any module whose
wasm2wat --enable-all output is larger than about 7.9 MB, although wasm2wat itself finishes in about 1 s on the
same machine. Output capture through the Windows owned-process path runs at about 262 KB/s, so the 30 s
per-command deadline always fires during WAT capture, well before the 32 MiB output budget is reached.

Measured with REA main, the WABT 1.0.42 windows-x64 release and synthetic modules (wall time includes CLI start-up):

WAT bytes	result	wall
4,016,497	ok	17.0 s
7,029,689	ok	28.7 s
8,034,539	`provider_timeout` (30000 ms), captured 7,843,840 chars	31.8 s
53,985,218 (MuJoCo 3.3.8 from `mujoco-js@0.0.7`, 8,047,277 B module)	`provider_timeout`, captured 7,802,880 chars	55.8 s

Running dist/windows/WindowsOwnedProcess.js directly on wasm2wat --enable-all for the 4 MB-WAT module gave
491 chunks, 490 of them exactly 8,192 B. The median gap between chunks was 31.27 ms, throughput was
258,563 B/s, and the run took 15.5 s in total. The same command through Node child_process.spawn with piped
stdio takes 133 ms. The 54 MB MuJoCo WAT takes 1.0 s that way.

Reproduction

On Windows x64, set REA_WABT_BIN_DIRECTORY=<abs>\wabt-1.0.42\bin (official 1.0.42 windows-x64 release).
Generate a WAT file with enough trivial functions that wasm2wat --enable-all of the compiled module is
about 8 MB, then compile it: wat2wasm synth.wat -o synth-8m.wasm.
Run rea inspect-wasm-artifact <abs>\synth-8m.wasm --format json.
Observe "code":"provider_timeout", timeout_ms: 30000, with about 7.8 MB of captured WAT. A module with about
7 MB of WAT succeeds in about 29 s.

Expected behavior

Output capture on Windows should reach throughput close to Node pipes, so that WAT capture for modules within the
documented limits (8 MiB input, 32 MiB output) finishes well inside the 30 s deadline, as wasm2wat alone does in
about 1 s.

Impact and scope

Affects inspect-wasm-artifact on Windows for any module above about 7.9 MB of WAT. Smaller modules succeed, with
digests matching and byte-identical JSON on rerun.
Evidence integrity looks fine: it fails closed with provider_timeout and no truncated success.
Other Windows owned-process consumers that produce large stdout probably see the same ~262 KB/s ceiling. That's
inferred from the shared code path; I only measured the WABT case.
Not tested on macOS or Linux.

Suspected cause (from reading source; not confirmed by instrumentation in the native code):
WindowsOwnedProcess schedules #poll with setTimeout(..., 25) (src/windows/WindowsOwnedProcess.ts:255-258),
which fires about every 31 ms on Windows. Each poll's native readOutput (native/windows/src/process.cc:216-229)
does one PeekNamedPipe plus a single ReadFile on a pipe created with the default buffer size
(CreatePipe(..., 0), line 86). So each poll gets at most one pipe buffer, which was 8,192 B in our measurements.
Possible fixes: drain the pipe in a loop within each poll (up to a per-poll cap), create the pipe with a larger
buffer, or use overlapped/IOCP reads.

Related notes (separate from this bug; happy to file separately if preferred):

WABT_LIMITS (src/wasm/WabtRelease.ts:7-12) is fixed (8 MiB input, 32 MiB output, 30 s) with no override, and WAT
is always generated. Even with a fast pipe, the MuJoCo module (54 MB WAT; 40 MB after wasm-strip) would fail the
output budget on every platform. A summary mode without WAT (validation, sections, import/export forms), or
configurable limits recorded in tool_profile, would help.
src/wasm/WabtArtifactProvider.ts:383 hard-codes "30-second deadline and 32 MiB" in limitations, so it would go
stale if the limits change.
With the limits raised locally (timeout 600 s, budget 128 MiB, nothing else changed), the MuJoCo module completed in
233 s. That matches the 262 KB/s model, and the result (19 sections, 119 imports, 23 exports, digest match) agreed
with standalone WABT.

REA version

main 67c7a24e60cc59191283977642055cd59a84b043 (rea --version → 6.4.0). The cited files are unchanged on main at
c02fb0ea.

Runtime and platform

Node v24.19.0; Windows 11 x64 (10.0.26200). Windows native bundle copied from the rea-agents@6.4.0 package, because
build:windows-native requires Linux with MinGW. The main-branch loader accepted it.

Target and artifact context

WebAssembly v1 modules: synthetic test modules and the Emscripten/embind MuJoCo 3.3.8 module embedded in mujoco-js@0.0.7.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

{"code":"provider_timeout","category":"timeout","message":"Analysis took too long...","details":{"operation":"inspect_wasm_artifact","timeout_ms":30000,"captured_output":{"stdout":"(module $mujoco_wasm.wasm ...

Investigated and drafted with Claude Code as part of a reverse-engineering research project; measurements were
taken on the hardware described above.

Originally created by @PixelPepper on GitHub (Oct 13, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/2722 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Process capture or runtime observation ### Observed behavior On Windows, `rea inspect-wasm-artifact` fails with `provider_timeout` (30000 ms) for any module whose `wasm2wat --enable-all` output is larger than about 7.9 MB, although `wasm2wat` itself finishes in about 1 s on the same machine. Output capture through the Windows owned-process path runs at about 262 KB/s, so the 30 s per-command deadline always fires during WAT capture, well before the 32 MiB output budget is reached. Measured with REA main, the WABT 1.0.42 windows-x64 release and synthetic modules (wall time includes CLI start-up): | WAT bytes | result | wall | |---|---|---| | 4,016,497 | ok | 17.0 s | | 7,029,689 | ok | 28.7 s | | 8,034,539 | `provider_timeout` (30000 ms), captured 7,843,840 chars | 31.8 s | | 53,985,218 (MuJoCo 3.3.8 from `mujoco-js@0.0.7`, 8,047,277 B module) | `provider_timeout`, captured 7,802,880 chars | 55.8 s | Running `dist/windows/WindowsOwnedProcess.js` directly on `wasm2wat --enable-all` for the 4 MB-WAT module gave 491 chunks, 490 of them exactly 8,192 B. The median gap between chunks was 31.27 ms, throughput was 258,563 B/s, and the run took 15.5 s in total. The same command through Node `child_process.spawn` with piped stdio takes 133 ms. The 54 MB MuJoCo WAT takes 1.0 s that way. ### Reproduction 1. On Windows x64, set `REA_WABT_BIN_DIRECTORY=<abs>\wabt-1.0.42\bin` (official 1.0.42 windows-x64 release). 2. Generate a WAT file with enough trivial functions that `wasm2wat --enable-all` of the compiled module is about 8 MB, then compile it: `wat2wasm synth.wat -o synth-8m.wasm`. 3. Run `rea inspect-wasm-artifact <abs>\synth-8m.wasm --format json`. 4. Observe `"code":"provider_timeout"`, `timeout_ms: 30000`, with about 7.8 MB of captured WAT. A module with about 7 MB of WAT succeeds in about 29 s. ### Expected behavior Output capture on Windows should reach throughput close to Node pipes, so that WAT capture for modules within the documented limits (8 MiB input, 32 MiB output) finishes well inside the 30 s deadline, as `wasm2wat` alone does in about 1 s. ### Impact and scope - Affects `inspect-wasm-artifact` on Windows for any module above about 7.9 MB of WAT. Smaller modules succeed, with digests matching and byte-identical JSON on rerun. - Evidence integrity looks fine: it fails closed with `provider_timeout` and no truncated success. - Other Windows owned-process consumers that produce large stdout probably see the same ~262 KB/s ceiling. That's inferred from the shared code path; I only measured the WABT case. - Not tested on macOS or Linux. **Suspected cause (from reading source; not confirmed by instrumentation in the native code):** `WindowsOwnedProcess` schedules `#poll` with `setTimeout(..., 25)` (`src/windows/WindowsOwnedProcess.ts:255-258`), which fires about every 31 ms on Windows. Each poll's native `readOutput` (`native/windows/src/process.cc:216-229`) does one `PeekNamedPipe` plus a single `ReadFile` on a pipe created with the default buffer size (`CreatePipe(..., 0)`, line 86). So each poll gets at most one pipe buffer, which was 8,192 B in our measurements. Possible fixes: drain the pipe in a loop within each poll (up to a per-poll cap), create the pipe with a larger buffer, or use overlapped/IOCP reads. **Related notes (separate from this bug; happy to file separately if preferred):** - `WABT_LIMITS` (`src/wasm/WabtRelease.ts:7-12`) is fixed (8 MiB input, 32 MiB output, 30 s) with no override, and WAT is always generated. Even with a fast pipe, the MuJoCo module (54 MB WAT; 40 MB after `wasm-strip`) would fail the output budget on every platform. A summary mode without WAT (validation, sections, import/export forms), or configurable limits recorded in `tool_profile`, would help. - `src/wasm/WabtArtifactProvider.ts:383` hard-codes "30-second deadline and 32 MiB" in `limitations`, so it would go stale if the limits change. - With the limits raised locally (timeout 600 s, budget 128 MiB, nothing else changed), the MuJoCo module completed in 233 s. That matches the 262 KB/s model, and the result (19 sections, 119 imports, 23 exports, digest match) agreed with standalone WABT. ### REA version main `67c7a24e60cc59191283977642055cd59a84b043` (`rea --version` → 6.4.0). The cited files are unchanged on main at `c02fb0ea`. ### Runtime and platform Node v24.19.0; Windows 11 x64 (10.0.26200). Windows native bundle copied from the rea-agents@6.4.0 package, because `build:windows-native` requires Linux with MinGW. The main-branch loader accepted it. ### Target and artifact context WebAssembly v1 modules: synthetic test modules and the Emscripten/embind MuJoCo 3.3.8 module embedded in `mujoco-js@0.0.7`. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence ```text {"code":"provider_timeout","category":"timeout","message":"Analysis took too long...","details":{"operation":"inspect_wasm_artifact","timeout_ms":30000,"captured_output":{"stdout":"(module $mujoco_wasm.wasm ... ``` --- _Investigated and drafted with Claude Code as part of a reverse-engineering research project; measurements were taken on the hardware described above._

bsakaguchi

2026-10-15 18:30:06 +00:00

closed this issue
added the
area: native

bug
labels

bsakaguchi commented

2026-10-15 18:30:07 +00:00

Author

Owner

@PixelPepper commented on GitHub (Oct 13, 2026):

Scope clarification.

The ≈7.9 MB WAT threshold is what was observed on this Windows host with REA main 67c7a24e plus the 6.4.0 native bundle. It isn't a universal module-size boundary: inputs above 8 MiB are rejected at admission first, and invalid modules or large wasm-objdump output can fail before WAT capture.
The measured reads were 8,192 bytes each, but the native readOutput permits up to 65,536 bytes per ReadFile (std::array<unsigned char, 65536>, min(available, 65536)). So the ≈262 KB/s ceiling describes this configuration, where each poll found one default-size pipe buffer available, rather than a hard-coded cap.
The separate 32 MiB output-budget obstruction for this module (54 MB WAT, 40 MB after wasm-strip) is inferred from source and WAT sizes. Linux/macOS weren't tested.

@PixelPepper commented on GitHub (Oct 13, 2026): **Scope clarification.** - The ≈7.9 MB WAT threshold is what was observed on this Windows host with REA main `67c7a24e` plus the 6.4.0 native bundle. It isn't a universal module-size boundary: inputs above 8 MiB are rejected at admission first, and invalid modules or large `wasm-objdump` output can fail before WAT capture. - The measured reads were 8,192 bytes each, but the native `readOutput` permits up to 65,536 bytes per `ReadFile` (`std::array<unsigned char, 65536>`, `min(available, 65536)`). So the ≈262 KB/s ceiling describes this configuration, where each poll found one default-size pipe buffer available, rather than a hard-coded cap. - The separate 32 MiB output-budget obstruction for this module (54 MB WAT, 40 MB after `wasm-strip`) is inferred from source and WAT sizes. Linux/macOS weren't tested.

bsakaguchi commented

2026-10-15 18:30:07 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Delivered on main in fffb07419.

The native Windows pipe drain already had the throughput correction, but lacked a portable executed regression at this workload size. Hosted Windows now verifies the actual owned-process/native capture path with more than 8 MiB of output, exact bytes/digest and verified cleanup within the 30-second workflow deadline.

Validation: Final hosted Windows receipt: 8,454,144 bytes in 360 ms; exact SHA-256 cd6b2297b720cc1b868f5479d51ce7e5f2e8259470193f8112dd2df8a73658c6. The final hosted Windows lane passed, including 449 curated tests and 25 prerequisite skips. The original WAT artifact was not available; this is a portable output-drain reproduction, not that exact target replay. Full exact-revision qualification passed.

@morluto commented on GitHub (Oct 14, 2026): Delivered on main in [fffb07419](https://github.com/morluto/rea/commit/fffb0741936067530d1ce9777461cd53e87398c2). The native Windows pipe drain already had the throughput correction, but lacked a portable executed regression at this workload size. Hosted Windows now verifies the actual owned-process/native capture path with more than 8 MiB of output, exact bytes/digest and verified cleanup within the 30-second workflow deadline. Validation: Final hosted Windows receipt: 8,454,144 bytes in 360 ms; exact SHA-256 cd6b2297b720cc1b868f5479d51ce7e5f2e8259470193f8112dd2df8a73658c6. The final hosted Windows lane passed, including 449 curated tests and 25 prerequisite skips. The original WAT artifact was not available; this is a portable output-drain reproduction, not that exact target replay. Full exact-revision [qualification](https://github.com/morluto/rea/actions/runs/38629313022) passed.

bsakaguchi referenced this issue

2026-10-17 21:07:29 +00:00

[PR #983] [MERGED] perf(reconstruction): index coverage evaluation #2553

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#983

No description provided.

Rows
Columns