morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5967] [Bug] Objdump and Rizin drop the output-loss flag from successful captures #3635

New issue

Closed

opened 2026-10-31 00:42:35 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:42:35 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5967

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The one-shot objdump/Rizin service discards a successful owned capture's diagnosticTruncated marker. In a focused test using the production collector, supervisor and service with simulated provider streams, a process reporting exit 0 and outputIncomplete: true produced a snapshot with diagnosticTruncated: true, but the resulting Evidence contained output_truncated: false and no partial-output limitation.

Both command services reproduced this projection. Matching exit-2 controls retained output_truncated: true, so the discrepancy is specific to the zero-exit return path. No real Windows output loss or real objdump/Rizin command was reproduced.

Reproduction

From the baseline checkout with locked dependencies installed, save the following as tests/composition/commandCaptureFacts.test.ts and run:

node node_modules/vitest/vitest.mjs run --project composition tests/composition/commandCaptureFacts.test.ts --maxWorkers=1 --no-file-parallelism

The eight characterization cases passed. They assert the current result, including the missing flag, rather than a proposed fix. The injected launcher supplies streams, an explicit output-loss marker, and simulated cleanup; the production runOwnedCommand constructs the returned snapshot.

import { EventEmitter } from "node:events";
import { PassThrough } from "node:stream";
import { expect, it } from "vitest";
import { createReverseEngineeringService } from "../../src/composition/reverseEngineering.js";
import { runOwnedCommand } from "../../src/process/OwnedCommand.js";

for (const tool of ["objdump", "rizin"] as const) {
  for (const [exitCode, incomplete] of [[0, false], [2, false], [0, true], [2, true]] as const) {
    it(`${tool}: exit=${exitCode}, outputIncomplete=${incomplete}`, async () => {
      let collected: Awaited<ReturnType<typeof runOwnedCommand>> | undefined;
      let cleanupCalls = 0;
      const service = createReverseEngineeringService({}, {
        runOwned: async (spawn, limits, options) => {
          collected = await runOwnedCommand(spawn, limits, {
            ...options,
            launcher: async () => {
              class SimulatedProcess extends EventEmitter {
                readonly stdout = new PassThrough();
                readonly stderr = new PassThrough();
                readonly pid = 41235;
                exitCode: number | null = null;
                signalCode: NodeJS.Signals | null = null;
                outputIncomplete = incomplete;
                kill() { return true; }
              }
              const child = new SimulatedProcess();
              setImmediate(() => {
                child.stdout.end("retained output");
                child.stderr.end();
                child.exitCode = exitCode;
                child.emit("exit", exitCode, null);
                child.emit("close", exitCode, null);
              });
              return {
                process: child,
                ownership: { runId: spawn.runId, leaderPid: child.pid, processGroupId: child.pid },
                cleanup: async () => { cleanupCalls++; return { cleaned: true, signaled: false }; },
              };
            },
          });
          return collected;
        },
      });
      try {
        const result = tool === "objdump"
          ? await service.inspectWithObjdump({ operation: "architectures" })
          : await service.executeRizinCommand({ path: "package.json", command: "iI" });
        if (!result.ok) throw result.error;
        expect(collected?.exitCode).toBe(exitCode);
        expect(collected?.diagnosticTruncated === true).toBe(incomplete);
        expect(cleanupCalls).toBe(1);
        const raw = result.value.raw_result as Record<string, unknown>;
        console.log(JSON.stringify({ tool, collectedExit: collected?.exitCode,
          collectedTruncated: collected?.diagnosticTruncated === true,
          result: raw, partialLimitation: result.value.limitations.some((s) => s.includes("captured output is partial")) }));
        expect(raw.exit_code).toBe(exitCode);
        // Characterize the observed defect: only zero exits lose the flag.
        expect(raw.output_truncated).toBe(incomplete && exitCode !== 0);
        expect(raw.completion_status).toBe("complete");
      } finally { await service.close(); }
    });
  }
}

Remove the temporary test after running it.

Expected behavior

Preserve the producer's explicit omitted-output fact as output_truncated: true and disclose partial capture in Evidence even when the process exits zero. The snapshot contract defines diagnosticTruncated as omitted bytes caused by the budget or provider output loss. The tool design contract requires reporting truncation and partial failures.

This report does not require changing the process exit status or treating every output-loss marker as unknown process completion. The documented completion_status guarantee specifically covers termination by buffer enforcement.

Impact and scope

P2: known omitted output is represented as untruncated Evidence, without the warning present on the equivalent nonzero-exit path. Consumers cannot distinguish that capture from a fully retained result using the advertised flag. The shared projection serves CLI and MCP, but only the application/collector boundary was executed for output loss.

The production Windows owner records output loss for destroyed/backpressured streams; that is source-derived reachability of the marker, not a measurement of actual objdump/Rizin loss. Ordinary backpressure alone is not established as a trigger. NOT_ENOUGH_INFO: real-provider frequency, a concrete Windows workload that triggers this marker, and a verified workaround through REA.

REA version

6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e. Relevant files are unchanged on origin/main at 25e1a24a3ed3671a0148757555f18d16c29d936a.

Runtime and platform

macOS 27.0.1 arm64; Node.js v26.12.0; Vitest 5.0.3. Simulated provider process streams; no native Windows execution.

Target and artifact context

Subjectless objdump architecture listing and Rizin against the checkout's unchanged package.json as an inert local artifact. The simulated executable does not inspect its contents.

Interface

Other

Earliest failure stage

Transport or result projection

Relevant output or evidence

Observed for both command services:

{"collectedExit":0,"collectedTruncated":true,"result":{"stdout":"retained output","stderr":"","exit_code":0,"signal":null,"output_truncated":false,"completion_status":"complete"},"partialLimitation":false}

The corresponding exit-2 controls returned exit_code: 2, output_truncated: true, and partialLimitation: true. Healthy controls reported false truncation as expected. The command status remained complete in all eight cases.

Typed outcome or recovery expected

Retain the owned snapshot's output-loss fact in the existing command observation and disclose its limitation. No new error type or execution authority is needed.

Analysis or suspected cause

Source-derived: the supervisor maps outputIncomplete into diagnosticTruncated. The collector can accept an otherwise successful snapshot with that marker. The adapter preserves it when projecting nonzero/signal failures, but omits it from its zero-exit return. The consumer then leaves outputTruncated at its initial false value; the limitation depends on that value.

The Windows producer records output loss at destroyed or discarded blocked streams and writes to a destroyed stream. The test models that explicit producer fact; it does not establish how often normal one-shot workloads reach it.

Additional context

Related #5896 concerns persistent Rizin debugger cleanup after protocol truncation; #2722/PR #2784 concern Windows capture throughput. PR #1960 changes reverse cleanup/session ownership and does not preserve this successful snapshot marker. These have different contracts and fixes.

NOT_ENOUGH_INFO: introducing commit and last known good version. No fix or restoration was tested.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5967 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The one-shot objdump/Rizin service discards a successful owned capture's `diagnosticTruncated` marker. In a focused test using the production collector, supervisor and service with simulated provider streams, a process reporting exit 0 and `outputIncomplete: true` produced a snapshot with `diagnosticTruncated: true`, but the resulting Evidence contained `output_truncated: false` and no partial-output limitation. Both command services reproduced this projection. Matching exit-2 controls retained `output_truncated: true`, so the discrepancy is specific to the zero-exit return path. No real Windows output loss or real objdump/Rizin command was reproduced. ### Reproduction From the baseline checkout with locked dependencies installed, save the following as `tests/composition/commandCaptureFacts.test.ts` and run: ```sh node node_modules/vitest/vitest.mjs run --project composition tests/composition/commandCaptureFacts.test.ts --maxWorkers=1 --no-file-parallelism ``` The eight characterization cases passed. They assert the current result, including the missing flag, rather than a proposed fix. The injected launcher supplies streams, an explicit output-loss marker, and simulated cleanup; the production `runOwnedCommand` constructs the returned snapshot. ```ts import { EventEmitter } from "node:events"; import { PassThrough } from "node:stream"; import { expect, it } from "vitest"; import { createReverseEngineeringService } from "../../src/composition/reverseEngineering.js"; import { runOwnedCommand } from "../../src/process/OwnedCommand.js"; for (const tool of ["objdump", "rizin"] as const) { for (const [exitCode, incomplete] of [[0, false], [2, false], [0, true], [2, true]] as const) { it(`${tool}: exit=${exitCode}, outputIncomplete=${incomplete}`, async () => { let collected: Awaited<ReturnType<typeof runOwnedCommand>> | undefined; let cleanupCalls = 0; const service = createReverseEngineeringService({}, { runOwned: async (spawn, limits, options) => { collected = await runOwnedCommand(spawn, limits, { ...options, launcher: async () => { class SimulatedProcess extends EventEmitter { readonly stdout = new PassThrough(); readonly stderr = new PassThrough(); readonly pid = 41235; exitCode: number | null = null; signalCode: NodeJS.Signals | null = null; outputIncomplete = incomplete; kill() { return true; } } const child = new SimulatedProcess(); setImmediate(() => { child.stdout.end("retained output"); child.stderr.end(); child.exitCode = exitCode; child.emit("exit", exitCode, null); child.emit("close", exitCode, null); }); return { process: child, ownership: { runId: spawn.runId, leaderPid: child.pid, processGroupId: child.pid }, cleanup: async () => { cleanupCalls++; return { cleaned: true, signaled: false }; }, }; }, }); return collected; }, }); try { const result = tool === "objdump" ? await service.inspectWithObjdump({ operation: "architectures" }) : await service.executeRizinCommand({ path: "package.json", command: "iI" }); if (!result.ok) throw result.error; expect(collected?.exitCode).toBe(exitCode); expect(collected?.diagnosticTruncated === true).toBe(incomplete); expect(cleanupCalls).toBe(1); const raw = result.value.raw_result as Record<string, unknown>; console.log(JSON.stringify({ tool, collectedExit: collected?.exitCode, collectedTruncated: collected?.diagnosticTruncated === true, result: raw, partialLimitation: result.value.limitations.some((s) => s.includes("captured output is partial")) })); expect(raw.exit_code).toBe(exitCode); // Characterize the observed defect: only zero exits lose the flag. expect(raw.output_truncated).toBe(incomplete && exitCode !== 0); expect(raw.completion_status).toBe("complete"); } finally { await service.close(); } }); } } ``` Remove the temporary test after running it. ### Expected behavior Preserve the producer's explicit omitted-output fact as `output_truncated: true` and disclose partial capture in Evidence even when the process exits zero. The [snapshot contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/process/ProviderProcess.ts#L110-L134) defines `diagnosticTruncated` as omitted bytes caused by the budget or provider output loss. The [tool design contract](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/tool-design.md#L47-L70) requires reporting truncation and partial failures. This report does not require changing the process exit status or treating every output-loss marker as unknown process completion. The documented `completion_status` guarantee specifically covers termination by buffer enforcement. ### Impact and scope P2: known omitted output is represented as untruncated Evidence, without the warning present on the equivalent nonzero-exit path. Consumers cannot distinguish that capture from a fully retained result using the advertised flag. The shared projection serves CLI and MCP, but only the application/collector boundary was executed for output loss. The production Windows owner records output loss for destroyed/backpressured streams; that is source-derived reachability of the marker, not a measurement of actual objdump/Rizin loss. Ordinary backpressure alone is not established as a trigger. NOT_ENOUGH_INFO: real-provider frequency, a concrete Windows workload that triggers this marker, and a verified workaround through REA. ### REA version 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. Relevant files are unchanged on origin/main at `25e1a24a3ed3671a0148757555f18d16c29d936a`. ### Runtime and platform macOS 27.0.1 arm64; Node.js v26.12.0; Vitest 5.0.3. Simulated provider process streams; no native Windows execution. ### Target and artifact context Subjectless objdump architecture listing and Rizin against the checkout's unchanged `package.json` as an inert local artifact. The simulated executable does not inspect its contents. ### Interface Other ### Earliest failure stage Transport or result projection ### Relevant output or evidence Observed for both command services: ```json {"collectedExit":0,"collectedTruncated":true,"result":{"stdout":"retained output","stderr":"","exit_code":0,"signal":null,"output_truncated":false,"completion_status":"complete"},"partialLimitation":false} ``` The corresponding exit-2 controls returned `exit_code: 2`, `output_truncated: true`, and `partialLimitation: true`. Healthy controls reported false truncation as expected. The command status remained `complete` in all eight cases. ### Typed outcome or recovery expected Retain the owned snapshot's output-loss fact in the existing command observation and disclose its limitation. No new error type or execution authority is needed. ### Analysis or suspected cause Source-derived: [the supervisor](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/process/ProviderProcess.ts#L419-L435) maps `outputIncomplete` into `diagnosticTruncated`. [The collector](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/process/OwnedCommand.ts#L330-L375) can accept an otherwise successful snapshot with that marker. [The adapter](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/reverse/ReverseEngineeringService.ts#L844-L875) preserves it when projecting nonzero/signal failures, but omits it from its zero-exit return. [The consumer](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/reverse/ReverseEngineeringService.ts#L390-L422) then leaves `outputTruncated` at its initial false value; [the limitation](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/reverse/ReverseEngineeringService.ts#L684-L688) depends on that value. The Windows producer records output loss at [destroyed or discarded blocked streams](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/windows/WindowsOwnedProcess.ts#L324-L336) and [writes to a destroyed stream](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/windows/WindowsOwnedProcess.ts#L395-L400). The test models that explicit producer fact; it does not establish how often normal one-shot workloads reach it. ### Additional context Related #5896 concerns persistent Rizin debugger cleanup after protocol truncation; #2722/PR #2784 concern Windows capture throughput. PR #1960 changes reverse cleanup/session ownership and does not preserve this successful snapshot marker. These have different contracts and fixes. NOT_ENOUGH_INFO: introducing commit and last known good version. No fix or restoration was tested.

bsakaguchi closed this issue

2026-10-31 00:42:35 +00:00

bsakaguchi commented

2026-10-31 00:42:36 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 24, 2026):

Checked the reported fix on main at d87e73f2db.

The owned-command adapter now copies diagnosticTruncated on the successful return, the consumer reads outputTruncated, and the shared raw/normalized command observation and limitation use that fact independently of completion status. The inspected parameterized test injects a successful truncated snapshot for both objdump and Rizin and explicitly asserts exit_code 0, output_truncated true, completion_status complete, and the partial-capture limitation. The test injects the owned snapshot rather than recreating the original production collector/supervisor stream-loss scenario; no real-provider or Windows behavior, test execution, CLI, or MCP runtime was verified.

Implementation: ReverseEngineeringService.ts, ReverseEngineeringService.ts.

The relevant regression checks in reverseEngineeringEdgeCases.test.ts passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

@coygeek commented on GitHub (Oct 24, 2026): Checked the reported fix on `main` at d87e73f2db50be72d33a4b3c7199af35518b441c. The owned-command adapter now copies diagnosticTruncated on the successful return, the consumer reads outputTruncated, and the shared raw/normalized command observation and limitation use that fact independently of completion status. The inspected parameterized test injects a successful truncated snapshot for both objdump and Rizin and explicitly asserts exit_code 0, output_truncated true, completion_status complete, and the partial-capture limitation. The test injects the owned snapshot rather than recreating the original production collector/supervisor stream-loss scenario; no real-provider or Windows behavior, test execution, CLI, or MCP runtime was verified. Implementation: [ReverseEngineeringService.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/reverse/ReverseEngineeringService.ts#L940), [ReverseEngineeringService.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/src/application/reverse/ReverseEngineeringService.ts#L479). The relevant regression checks in [reverseEngineeringEdgeCases.test.ts](https://github.com/morluto/rea/blob/d87e73f2db50be72d33a4b3c7199af35518b441c/tests/composition/reverseEngineeringEdgeCases.test.ts) passed in the focused test run on this revision. This closure is based on the current source and those checks; provider fixtures do not establish a real-engine run. Closing as completed on main.

bsakaguchi referenced this issue

2026-10-31 01:10:00 +00:00

[PR #3639] [CLOSED] fix(setup): preserve overridden runner variables and test isolation on Windows #5861

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3635

No description provided.

Rows
Columns