morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #1128] fix(har): heap exhaustion returns unrelated retry and doctor advice #401

New issue

Closed

opened 2026-10-15 18:24:46 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:24:46 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 8, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/1128

Summary

When inspect-web-network-capture exhausts its documented HAR child heap, its message and remediation.action tell the caller to retry and run rea doctor, while retryable is false. The detailed diagnostics preserve V8 heap exhaustion, but the recovery advice does not address the exhausted resource. Both the published release and the current source build reproduce this guidance on a synthetic HAR 1.2 capture of 33,552,514 bytes containing 34,098 entries with 512-byte response bodies. The input is below the 33,554,432-byte ceiling and its complete decoder reply measures 76,050,800 bytes, below the 100,663,296-byte reply ceiling. The independent heap cap still permits a bounded resource failure; this report concerns its misleading recovery advice.

Steps to reproduce

Install the published baseline from Runtime or environment into a disposable prefix. HAR inspection needs no external browser, proxy or analysis engine.

work=$(mktemp -d)
npm install --global --prefix "$work/install" rea-agents@6.0.0

Save this fixture generator as generate.py in that directory and run python3 "$work/generate.py". It writes a tiny control, a half-limit control and the failing near-limit capture. All URLs are synthetic and this command inspects captures offline.

import json
from pathlib import Path

root = Path(__file__).resolve().parent
entry = {
    'startedDateTime': '2026-01-01T00:00:00.000Z', 'time': 1,
    'request': {'method': 'GET', 'url': 'https://example.invalid/data',
                'httpVersion': 'HTTP/1.1', 'cookies': [], 'headers': [],
                'queryString': [], 'headersSize': -1, 'bodySize': 0},
    'response': {'status': 200, 'statusText': 'OK', 'httpVersion': 'HTTP/1.1',
                 'cookies': [], 'headers': [],
                 'content': {'size': 512, 'mimeType': 'text/plain', 'text': 'x' * 512},
                 'redirectURL': '', 'headersSize': -1, 'bodySize': 512},
    'cache': {}, 'timings': {'send': 0, 'wait': 1, 'receive': 0}
}
prefix = '{"log":{"version":"1.2","creator":{"name":"synthetic","version":"1"},"entries":['
suffix = ']}}'
record = json.dumps(entry, separators=(',', ':'))
for name, target in [('tiny', 3000), ('half', 16 * 1024 * 1024),
                     ('near', 32 * 1024 * 1024 - 1024)]:
    count = max(1, (target - len(prefix) - len(suffix) + 1) // (len(record) + 1))
    data = (prefix + ','.join([record] * count) + suffix).encode()
    (root / (name + '.har')).write_bytes(data)
    print(name, len(data), 'bytes;', count, 'entries')

Run the CLI with the explicit format and absolute capture path. The generated directory from mktemp -d is absolute on the tested platform.

"$work/install/bin/rea" inspect-web-network-capture "$work/tiny.har" har --json > "$work/tiny.json"
"$work/install/bin/rea" inspect-web-network-capture "$work/half.har" har --json > "$work/half.json"
"$work/install/bin/rea" inspect-web-network-capture "$work/near.har" har --json > "$work/near.json"
echo "$?"

The first two commands exit 0; the near-limit command exits 1. For the source baseline, the same three invocations were executed through node scripts/rea.mjs after a fresh locked dependency install and npm run build:cached. These are known reproductions, not a claim of a minimal fixture.

Expected behavior

When the independent heap budget prevents completion, the message and recovery action should explain the exhausted resource and give applicable guidance for the capture workload. They should agree with the result's retryability. The guide explicitly documents a bounded heap and permits resource failures, so being below the input and reply byte limits alone is not a guarantee of successful decoding. The expected restoration is accurate recovery guidance for this supported bounded failure, rather than a requirement to decode every under-limit capture.

Actual behavior

Each tested build succeeded once on the tiny fixture and once on the half-limit fixture. Successful replies retained all 2 and 17,049 records respectively. The near-limit fixture failed on both release attempts and on the single source-build attempt. The first release failure took 4.39 seconds and the source failure took 6.59 seconds, below the 30-second decoder deadline. The second release attempt followed the advised retry and again returned execution_failure, SIGABRT and V8 heap exhaustion. The following excerpt preserves the relevant fields; stack addresses, process IDs and executable paths are omitted.

code: execution_failure
category: execution_failure
retryable: false
message: Analysis could not complete. Retry once; if it continues, run `rea doctor`.
remediation.action: Analysis could not complete. Retry once; if it continues, run `rea doctor`.
details.provider_id: har
details.operation: inspect_web_network_capture
details.diagnostics.phase: decoder
details.diagnostics.exit_code: null
details.diagnostics.signal: SIGABRT
details.diagnostics.reason: Command failed; inspect its exit status and diagnostics.
details.diagnostics.stderr:
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

V8's final garbage-collection diagnostics report approximately 191–192 MB of heap, and diagnostic_truncated is false. The CLI process survived, returned failure JSON and produced no successful capture evidence. The resource reason is preserved in stderr; the defect is the top-level recovery advice. This is not a claim of parent-process OOM or a security boundary violation.

Following the other advised action, the source build's rea doctor --json reported checks named node, host, hopper, ghidra, ida-registration and skill:identity. It supplied no HAR workload or heap check and no HAR memory recovery action. The doctor diagnostics implementation likewise collects host, analysis-provider and optional-target health rather than the failed HAR resource budget.

Evidence

Expected source: The release capture guide documents 32 MiB input, 96 MiB decoder reply, 64-level nesting, 192 MiB old-generation heap and 30-second deadline. The current boundary guidance requires diagnostics identifying the failed constraint and recovery advice addressing its reason.
Failure source: The executed CLI reproduction above on both tested builds returned the shown execution_failure, SIGABRT and V8 heap-exhaustion diagnostic.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Fixture sizes, CLI exit codes, structured failures and successful controls were directly measured; the limits and implementation comparison are source-derived.

The exact fixture sizes are 2,050 bytes for 2 entries, 16,776,298 bytes for 17,049 entries, and 33,552,514 bytes for 34,098 entries. The failing input is 1,918 bytes below the enforced inclusive input ceiling.

To distinguish invalid HAR or oversized reply from heap capacity, a separate diagnostic process directly invoked the release's unchanged decodeHarCapture(text, []) with a 1,024 MiB old-generation heap. It passed the upstream HAR schema and returned all 34,098 records. Buffer.byteLength(JSON.stringify({ok: true, value})) measured exactly 76,050,800 bytes. This process measured validity and the decoder reply only; it was not a successful ordinary CLI run and is not a supported workaround. The corresponding diagnostic can be rerun after step 2:

node --max-old-space-size=1024 --input-type=module - "$work" <<'JS'
import {readFileSync} from 'node:fs';
import {pathToFileURL} from 'node:url';
const root = process.argv[2];
const moduleUrl = pathToFileURL(root + '/install/lib/node_modules/rea-agents/dist/browser/history/HarCapture.js');
const {decodeHarCapture} = await import(moduleUrl.href);
const value = decodeHarCapture(readFileSync(root + '/near.har', 'utf8'), []);
console.log({records: value.total_records,
             replyBytes: Buffer.byteLength(JSON.stringify({ok: true, value}))});
JS

The source adapter still launches the decoder with --max-old-space-size=192. The historical decoder enforces the input budget and removes inherited NODE_OPTIONS, so setting that variable does not override this child heap. The reply writer serializes the complete reply before checking its byte size. These are source observations, not proof of the exact allocation causing this abort.

Restoration check

Run the same ordinary CLI commands against the fixture generator. If the near-limit workload reaches the heap cap, verify that its message and recovery action identify the resource constraint, give applicable guidance and agree with retryability. Successful complete decoding with all 34,098 records also eliminates this failure path. Tiny and half-limit controls must keep succeeding with all original records, and oversized inputs must retain the documented input rejection. Preserve bounded execution and complete-evidence semantics; a partial successful result does not restore this contract.

Runtime or environment

Published baseline: npm rea-agents 6.0.0, tag rea-agents-6.0.0, commit 6fee42689ae6e95a0182e0d4d0f55644e73e72be.
Source baseline: main at bc4aa46d7c3960a1f325523e48aed2089b176e84, built from a fresh clone with locked dependencies. The default-branch ref was checked again after the build.
Runtime and platform: Node.js 22.23.2, npm 12.0.2, macOS 27.0 arm64. Both CLI builds ran with an isolated home and private temporary directory. No external engine was required.
Area: Evidence, artifacts, or reconstruction. Interface: CLI. Earliest failure stage: HAR decoder during artifact inspection. Target: locally generated HAR 1.2, repeated valid GET entries and synthetic ASCII response bodies.

Additional context

The tag-to-source comparison leaves the HAR adapter, decoder process, capture projection, input/reply constants and dependency lockfile unchanged. The parser's intervening change preserves prototype-named JSON members; these fixtures contain none. A different fixture with 2,048-byte response bodies and 13,304 entries decoded successfully at 33,552,770 bytes on the release, so failure depends on workload structure, not input bytes alone.

Open and closed issues and pull requests were searched, including HAR, OOM, heap, decoder and resource-limit discussions. The original capture PR contains a native Python MemoryError classification fix, which does not cover this HAR V8 abort. The open transport-ceiling PR concerns MCP result serialization and does not change the HAR heap or child reply writer. No duplicate report of this failure was found. The tracker search and privacy checks requested by the bug-report form were completed. No attachments are needed because the fixture and failure excerpt are inline.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: In both observed release heap-limit failures and the source-build failure, recovery advice directs the caller to retry and run doctor rather than address the capture's memory constraint, despite declaring the failure non-retryable. The instructed retry failed the same way, and doctor supplied no HAR resource check or recovery action. This is consequential failure guidance for the HAR inspection workflow. The documented bounded failure itself, input integrity and process containment are not assessed as defects.
Workaround status: unverified
Workaround basis: Smaller fixtures succeed, but preserving the original capture's complete evidence and coordinates through partitioning has not been verified. Raising the diagnostic process heap bypasses the supported child command, and inherited NODE_OPTIONS is removed. No supported complete-result workaround was demonstrated.

Originally created by @coygeek on GitHub (Oct 8, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/1128 ## Summary When `inspect-web-network-capture` exhausts its documented HAR child heap, its message and `remediation.action` tell the caller to retry and run `rea doctor`, while `retryable` is false. The detailed diagnostics preserve V8 heap exhaustion, but the recovery advice does not address the exhausted resource. Both the published release and the current source build reproduce this guidance on a synthetic HAR 1.2 capture of 33,552,514 bytes containing 34,098 entries with 512-byte response bodies. The input is below the 33,554,432-byte ceiling and its complete decoder reply measures 76,050,800 bytes, below the 100,663,296-byte reply ceiling. The independent heap cap still permits a bounded resource failure; this report concerns its misleading recovery advice. ## Steps to reproduce 1. Install the published baseline from Runtime or environment into a disposable prefix. HAR inspection needs no external browser, proxy or analysis engine. ```sh work=$(mktemp -d) npm install --global --prefix "$work/install" rea-agents@6.0.0 ``` 2. Save this fixture generator as `generate.py` in that directory and run `python3 "$work/generate.py"`. It writes a tiny control, a half-limit control and the failing near-limit capture. All URLs are synthetic and this command inspects captures offline. ```python import json from pathlib import Path root = Path(__file__).resolve().parent entry = { 'startedDateTime': '2026-01-01T00:00:00.000Z', 'time': 1, 'request': {'method': 'GET', 'url': 'https://example.invalid/data', 'httpVersion': 'HTTP/1.1', 'cookies': [], 'headers': [], 'queryString': [], 'headersSize': -1, 'bodySize': 0}, 'response': {'status': 200, 'statusText': 'OK', 'httpVersion': 'HTTP/1.1', 'cookies': [], 'headers': [], 'content': {'size': 512, 'mimeType': 'text/plain', 'text': 'x' * 512}, 'redirectURL': '', 'headersSize': -1, 'bodySize': 512}, 'cache': {}, 'timings': {'send': 0, 'wait': 1, 'receive': 0} } prefix = '{"log":{"version":"1.2","creator":{"name":"synthetic","version":"1"},"entries":[' suffix = ']}}' record = json.dumps(entry, separators=(',', ':')) for name, target in [('tiny', 3000), ('half', 16 * 1024 * 1024), ('near', 32 * 1024 * 1024 - 1024)]: count = max(1, (target - len(prefix) - len(suffix) + 1) // (len(record) + 1)) data = (prefix + ','.join([record] * count) + suffix).encode() (root / (name + '.har')).write_bytes(data) print(name, len(data), 'bytes;', count, 'entries') ``` 3. Run the CLI with the explicit format and absolute capture path. The generated directory from `mktemp -d` is absolute on the tested platform. ```sh "$work/install/bin/rea" inspect-web-network-capture "$work/tiny.har" har --json > "$work/tiny.json" "$work/install/bin/rea" inspect-web-network-capture "$work/half.har" har --json > "$work/half.json" "$work/install/bin/rea" inspect-web-network-capture "$work/near.har" har --json > "$work/near.json" echo "$?" ``` The first two commands exit 0; the near-limit command exits 1. For the source baseline, the same three invocations were executed through `node scripts/rea.mjs` after a fresh locked dependency install and `npm run build:cached`. These are known reproductions, not a claim of a minimal fixture. ## Expected behavior When the independent heap budget prevents completion, the message and recovery action should explain the exhausted resource and give applicable guidance for the capture workload. They should agree with the result's retryability. The guide explicitly documents a bounded heap and permits resource failures, so being below the input and reply byte limits alone is not a guarantee of successful decoding. The expected restoration is accurate recovery guidance for this supported bounded failure, rather than a requirement to decode every under-limit capture. ## Actual behavior Each tested build succeeded once on the tiny fixture and once on the half-limit fixture. Successful replies retained all 2 and 17,049 records respectively. The near-limit fixture failed on both release attempts and on the single source-build attempt. The first release failure took 4.39 seconds and the source failure took 6.59 seconds, below the 30-second decoder deadline. The second release attempt followed the advised retry and again returned `execution_failure`, `SIGABRT` and V8 heap exhaustion. The following excerpt preserves the relevant fields; stack addresses, process IDs and executable paths are omitted. ```text code: execution_failure category: execution_failure retryable: false message: Analysis could not complete. Retry once; if it continues, run `rea doctor`. remediation.action: Analysis could not complete. Retry once; if it continues, run `rea doctor`. details.provider_id: har details.operation: inspect_web_network_capture details.diagnostics.phase: decoder details.diagnostics.exit_code: null details.diagnostics.signal: SIGABRT details.diagnostics.reason: Command failed; inspect its exit status and diagnostics. details.diagnostics.stderr: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory ``` V8's final garbage-collection diagnostics report approximately 191–192 MB of heap, and `diagnostic_truncated` is false. The CLI process survived, returned failure JSON and produced no successful capture evidence. The resource reason is preserved in stderr; the defect is the top-level recovery advice. This is not a claim of parent-process OOM or a security boundary violation. Following the other advised action, the source build's `rea doctor --json` reported checks named `node`, `host`, `hopper`, `ghidra`, `ida-registration` and `skill:identity`. It supplied no HAR workload or heap check and no HAR memory recovery action. The [doctor diagnostics implementation](https://github.com/morluto/rea/blob/bc4aa46d7c3960a1f325523e48aed2089b176e84/src/application/DoctorDiagnostics.ts#L26) likewise collects host, analysis-provider and optional-target health rather than the failed HAR resource budget. ## Evidence - Expected source: The [release capture guide](https://github.com/morluto/rea/blob/6fee42689ae6e95a0182e0d4d0f55644e73e72be/docs/web-network-captures.md#L115) documents 32 MiB input, 96 MiB decoder reply, 64-level nesting, 192 MiB old-generation heap and 30-second deadline. The [current boundary guidance](https://github.com/morluto/rea/blob/bc4aa46d7c3960a1f325523e48aed2089b176e84/AGENTS.md#L61) requires diagnostics identifying the failed constraint and recovery advice addressing its reason. - Failure source: The executed CLI reproduction above on both tested builds returned the shown `execution_failure`, `SIGABRT` and V8 heap-exhaustion diagnostic. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete Fixture sizes, CLI exit codes, structured failures and successful controls were directly measured; the limits and implementation comparison are source-derived. The exact fixture sizes are 2,050 bytes for 2 entries, 16,776,298 bytes for 17,049 entries, and 33,552,514 bytes for 34,098 entries. The failing input is 1,918 bytes below the enforced inclusive input ceiling. To distinguish invalid HAR or oversized reply from heap capacity, a separate diagnostic process directly invoked the release's unchanged `decodeHarCapture(text, [])` with a 1,024 MiB old-generation heap. It passed the upstream HAR schema and returned all 34,098 records. `Buffer.byteLength(JSON.stringify({ok: true, value}))` measured exactly 76,050,800 bytes. This process measured validity and the decoder reply only; it was not a successful ordinary CLI run and is not a supported workaround. The corresponding diagnostic can be rerun after step 2: ```sh node --max-old-space-size=1024 --input-type=module - "$work" <<'JS' import {readFileSync} from 'node:fs'; import {pathToFileURL} from 'node:url'; const root = process.argv[2]; const moduleUrl = pathToFileURL(root + '/install/lib/node_modules/rea-agents/dist/browser/history/HarCapture.js'); const {decodeHarCapture} = await import(moduleUrl.href); const value = decodeHarCapture(readFileSync(root + '/near.har', 'utf8'), []); console.log({records: value.total_records, replyBytes: Buffer.byteLength(JSON.stringify({ok: true, value}))}); JS ``` The [source adapter](https://github.com/morluto/rea/blob/bc4aa46d7c3960a1f325523e48aed2089b176e84/src/browser/history/HarCaptureAdapter.ts#L9) still launches the decoder with `--max-old-space-size=192`. The [historical decoder](https://github.com/morluto/rea/blob/bc4aa46d7c3960a1f325523e48aed2089b176e84/src/browser/history/HistoricalCaptureDecoder.ts#L83) enforces the input budget and removes inherited `NODE_OPTIONS`, so setting that variable does not override this child heap. The [reply writer](https://github.com/morluto/rea/blob/bc4aa46d7c3960a1f325523e48aed2089b176e84/src/browser/history/HarCaptureProcess.ts#L54) serializes the complete reply before checking its byte size. These are source observations, not proof of the exact allocation causing this abort. ## Restoration check Run the same ordinary CLI commands against the fixture generator. If the near-limit workload reaches the heap cap, verify that its message and recovery action identify the resource constraint, give applicable guidance and agree with retryability. Successful complete decoding with all 34,098 records also eliminates this failure path. Tiny and half-limit controls must keep succeeding with all original records, and oversized inputs must retain the documented input rejection. Preserve bounded execution and complete-evidence semantics; a partial successful result does not restore this contract. ## Runtime or environment - Published baseline: npm `rea-agents` 6.0.0, tag `rea-agents-6.0.0`, commit `6fee42689ae6e95a0182e0d4d0f55644e73e72be`. - Source baseline: `main` at `bc4aa46d7c3960a1f325523e48aed2089b176e84`, built from a fresh clone with locked dependencies. The default-branch ref was checked again after the build. - Runtime and platform: Node.js 22.23.2, npm 12.0.2, macOS 27.0 arm64. Both CLI builds ran with an isolated home and private temporary directory. No external engine was required. - Area: Evidence, artifacts, or reconstruction. Interface: CLI. Earliest failure stage: HAR decoder during artifact inspection. Target: locally generated HAR 1.2, repeated valid GET entries and synthetic ASCII response bodies. ## Additional context The tag-to-source comparison leaves the HAR adapter, decoder process, capture projection, input/reply constants and dependency lockfile unchanged. The parser's intervening change preserves prototype-named JSON members; these fixtures contain none. A different fixture with 2,048-byte response bodies and 13,304 entries decoded successfully at 33,552,770 bytes on the release, so failure depends on workload structure, not input bytes alone. Open and closed issues and pull requests were searched, including HAR, OOM, heap, decoder and resource-limit discussions. The [original capture PR](https://github.com/morluto/rea/pull/992) contains a native Python `MemoryError` classification fix, which does not cover this HAR V8 abort. The [open transport-ceiling PR](https://github.com/morluto/rea/pull/1101) concerns MCP result serialization and does not change the HAR heap or child reply writer. No duplicate report of this failure was found. The tracker search and privacy checks requested by the bug-report form were completed. No attachments are needed because the fixture and failure excerpt are inline. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: In both observed release heap-limit failures and the source-build failure, recovery advice directs the caller to retry and run doctor rather than address the capture's memory constraint, despite declaring the failure non-retryable. The instructed retry failed the same way, and doctor supplied no HAR resource check or recovery action. This is consequential failure guidance for the HAR inspection workflow. The documented bounded failure itself, input integrity and process containment are not assessed as defects. - Workaround status: unverified - Workaround basis: Smaller fixtures succeed, but preserving the original capture's complete evidence and coordinates through partitioning has not been verified. Raising the diagnostic process heap bypasses the supported child command, and inherited NODE_OPTIONS is removed. No supported complete-result workaround was demonstrated.

bsakaguchi

2026-10-15 18:24:46 +00:00

closed this issue
added the
area: runtime

bug
labels

bsakaguchi commented

2026-10-15 18:24:47 +00:00

Author

Owner

@N0zoM1z0 commented on GitHub (Oct 9, 2026):

Verified the fix in 44fd18305f against the report's exact generated workloads on Linux/Node 24.18.0: tiny (2 entries) and half-limit (17,049 entries) return complete results; near-limit (33,552,514 bytes / 34,098 entries) returns resource_constraint, memory, retryable=false, retained SIGABRT/V8 diagnostics, and applicable smaller-export/alternate-decoder advice in about 9 seconds. The 33,554,433-byte input remains rejected as out of range. No partial success or heap increase was used. Main CI passed: https://github.com/morluto/rea/actions/runs/37862776952 .

@N0zoM1z0 commented on GitHub (Oct 9, 2026): Verified the fix in 44fd18305fa5a8893ebb5b952099fdf45dbd6e55 against the report's exact generated workloads on Linux/Node 24.18.0: tiny (2 entries) and half-limit (17,049 entries) return complete results; near-limit (33,552,514 bytes / 34,098 entries) returns `resource_constraint`, memory, retryable=false, retained SIGABRT/V8 diagnostics, and applicable smaller-export/alternate-decoder advice in about 9 seconds. The 33,554,433-byte input remains rejected as out of range. No partial success or heap increase was used. Main CI passed: https://github.com/morluto/rea/actions/runs/37862776952 .

bsakaguchi referenced this issue

2026-10-16 21:12:49 +00:00

[PR #401] [MERGED] chore(main): release rea-agents 2.3.0 #1954

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#401

No description provided.

Rows
Columns