morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5055] [Bug] LLDB structured readback reports replacement-decoded malformed UTF-8 as facts #3099

New issue

Closed

opened 2026-10-31 00:36:37 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:36:37 +00:00

Owner

Originally created by @morluto on GitHub (Oct 18, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5055

Before submitting

Searched open/closed LLDB UTF-8, journal and readback issues and related PRs; #4742 concerns descriptor ownership, not decoding.
Only inert generated JSON byte fixtures are used.

Area

Evidence, artifacts, or reconstruction

Observed behavior

At main 9e259f87c8be3d22a0b96f12ed7838830869c987, parseLldbObservationJournal decodes the complete byte stream with replacement semantics before parsing its JSONL records. Invalid UTF-8 inside an otherwise valid JSON string becomes a literal U+FFFD and is published as a factual stop reason without a malformed-record limitation.

Reproduction

Executed a small boundary regression under Node 24.18.0 on macOS arm64. Supply completed journal records containing these bytes between valid event records:

Buffer.concat([
  Buffer.from('{"kind":"other-stop","reason":"before'),
  Buffer.from([0xff]),
  Buffer.from('after"}\n'),
]);

Baseline: 1 failed / 7 passed; actual parsed stop was before�after. A separate valid UTF-8 record containing an intentional U+FFFD remains valid and must not be rejected by character-pattern matching. These are executed production-parser controls, not an LLDB/device runtime reproduction.

Expected behavior

Interpret byte encoding at the structured readback owner before accepting JSON facts. Reject only the malformed complete record with the existing bounded limitation; retain valid neighboring records, intentional replacement characters, and the existing distinction for a torn final record. A complete tracer JSON result has the same encoding boundary and must not turn malformed source bytes into a reported error or observation string.

Impact and scope

Verified on generated byte fixtures at the journal parser; the complete result reader uses the same replacement-decoding mechanism by source inspection. A local controlled production-tracer regression covers invalid result bytes with valid retained journal facts. No native LLDB or target process was launched; file readback and cleanup use the production owner. Real-world prevalence, ordinary corruption, Linux/Windows execution and complete agent workflows: NOT_ENOUGH_INFO.

REA version / target / interface

Source 9e259f87c8be3d22a0b96f12ed7838830869c987, package 6.7.0. Generated JSONL and JSON files; provider boundary.

Earliest failure stage / evidence

Native analysis or reconstruction: structured trace readback before Evidence projection. src/native/LldbRetainedObservations.ts uses Buffer.from(bytes).toString("utf8"); src/native/LldbCallTracer.ts uses readFile("utf8") before JSON validation. Both replace malformed bytes before validation. The existing parser promises to retain valid records around malformed ones; rejecting the entire journal would lose useful facts.

Node 24's Buffer and TextDecoder documentation was checked through Context7 and DeepWiki, then against bounded runtime controls. Byte validity must be checked directly; presence of U+FFFD does not identify invalid source bytes. A local candidate decodes complete byte-delimited rows strictly and preserves structured-result decoding/JSON/schema failure causes through the existing fallback. The final selected LLDB check passed 28 tests with one skipped; the pre-existing heavy 8 MiB malformed-row case was excluded. No performance improvement is claimed.

Originally created by @morluto on GitHub (Oct 18, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5055 ### Before submitting - [x] Searched open/closed LLDB UTF-8, journal and readback issues and related PRs; #4742 concerns descriptor ownership, not decoding. - [x] Only inert generated JSON byte fixtures are used. ### Area Evidence, artifacts, or reconstruction ### Observed behavior At main `9e259f87c8be3d22a0b96f12ed7838830869c987`, `parseLldbObservationJournal` decodes the complete byte stream with replacement semantics before parsing its JSONL records. Invalid UTF-8 inside an otherwise valid JSON string becomes a literal U+FFFD and is published as a factual stop reason without a malformed-record limitation. ### Reproduction Executed a small boundary regression under Node 24.18.0 on macOS arm64. Supply completed journal records containing these bytes between valid event records: ```js Buffer.concat([ Buffer.from('{"kind":"other-stop","reason":"before'), Buffer.from([0xff]), Buffer.from('after"}\n'), ]); ``` Baseline: 1 failed / 7 passed; actual parsed stop was `before�after`. A separate valid UTF-8 record containing an intentional U+FFFD remains valid and must not be rejected by character-pattern matching. These are executed production-parser controls, not an LLDB/device runtime reproduction. ### Expected behavior Interpret byte encoding at the structured readback owner before accepting JSON facts. Reject only the malformed complete record with the existing bounded limitation; retain valid neighboring records, intentional replacement characters, and the existing distinction for a torn final record. A complete tracer JSON result has the same encoding boundary and must not turn malformed source bytes into a reported error or observation string. ### Impact and scope Verified on generated byte fixtures at the journal parser; the complete result reader uses the same replacement-decoding mechanism by source inspection. A local controlled production-tracer regression covers invalid result bytes with valid retained journal facts. No native LLDB or target process was launched; file readback and cleanup use the production owner. Real-world prevalence, ordinary corruption, Linux/Windows execution and complete agent workflows: NOT_ENOUGH_INFO. ### REA version / target / interface Source `9e259f87c8be3d22a0b96f12ed7838830869c987`, package 6.7.0. Generated JSONL and JSON files; provider boundary. ### Earliest failure stage / evidence Native analysis or reconstruction: structured trace readback before Evidence projection. `src/native/LldbRetainedObservations.ts` uses `Buffer.from(bytes).toString("utf8")`; `src/native/LldbCallTracer.ts` uses `readFile("utf8")` before JSON validation. Both replace malformed bytes before validation. The existing parser promises to retain valid records around malformed ones; rejecting the entire journal would lose useful facts. Node 24's Buffer and TextDecoder documentation was checked through Context7 and DeepWiki, then against bounded runtime controls. Byte validity must be checked directly; presence of U+FFFD does not identify invalid source bytes. A local candidate decodes complete byte-delimited rows strictly and preserves structured-result decoding/JSON/schema failure causes through the existing fallback. The final selected LLDB check passed 28 tests with one skipped; the pre-existing heavy 8 MiB malformed-row case was excluded. No performance improvement is claimed.

bsakaguchi

2026-10-31 00:36:37 +00:00

closed this issue
added the
triage

bug
labels

bsakaguchi commented

2026-10-31 00:36:38 +00:00

Author

Owner

@morluto commented on GitHub (Oct 18, 2026):

Fixed on main in 7201e8a44. Root cause: lossy byte decoding happened before structured validation. The record owner now strictly decodes each completed byte-delimited row; malformed rows produce the existing bounded limitation while valid neighbors, intentional U+FFFD and torn-tail semantics survive. Complete result readback retains physical/decode/JSON/schema failure causes instead of collapsing them into missing output; schema diagnostics summarize issue count while the full cause remains available.

Validation: selected LLDB boundary tests 28 passed / one large allocation case skipped; combined native selection 68 passed / two skipped in five files. Full npm run check, independent review and pre-push checks passed. Controlled production tracer/readback fixtures only; no actual LLDB/target execution. Bounded completed-task warm medians: reply token estimate 307 → 307, elapsed 0.1706 → 0.1714 s, CPU 0.2655 → 0.2682 s, sampled family RSS 188.4 → 194.6 MB. Two warm repeats/build plus separate first processes; no true cold-cache or speedup claim. Whole-agent/provider/platform costs remain unmeasured. Adjacent PID read limits and partial-prefix error retention remain separate source candidates requiring probes, not established defects in this fix.

@morluto commented on GitHub (Oct 18, 2026): Fixed on main in 7201e8a44. Root cause: lossy byte decoding happened before structured validation. The record owner now strictly decodes each completed byte-delimited row; malformed rows produce the existing bounded limitation while valid neighbors, intentional U+FFFD and torn-tail semantics survive. Complete result readback retains physical/decode/JSON/schema failure causes instead of collapsing them into missing output; schema diagnostics summarize issue count while the full cause remains available. Validation: selected LLDB boundary tests 28 passed / one large allocation case skipped; combined native selection 68 passed / two skipped in five files. Full `npm run check`, independent review and pre-push checks passed. Controlled production tracer/readback fixtures only; no actual LLDB/target execution. Bounded completed-task warm medians: reply token estimate 307 → 307, elapsed 0.1706 → 0.1714 s, CPU 0.2655 → 0.2682 s, sampled family RSS 188.4 → 194.6 MB. Two warm repeats/build plus separate first processes; no true cold-cache or speedup claim. Whole-agent/provider/platform costs remain unmeasured. Adjacent PID read limits and partial-prefix error retention remain separate source candidates requiring probes, not established defects in this fix.

bsakaguchi referenced this issue

2026-10-31 01:07:59 +00:00

[PR #3099] [CLOSED] fix(frida): attach by PID when process lookup omits target #5614

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3099

No description provided.

Rows
Columns