morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5837] [Bug] Artifact inspection reports complete coverage when its inventory traversal is unknown #3534

New issue

Closed

opened 2026-10-31 00:41:35 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:41:35 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5837

Originally assigned to: @kaoru0822-kitauji on GitHub.

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Executed through the production MCP server with a local application directory containing ordinary files and a FIFO. After open_binary, inspect_artifact succeeded. Its nested inventory reported coverage: {scope: "container-entries", status: "unknown"} and named the omitted FIFO. The enclosing inspection nevertheless reported coverage.status: "complete-within-substeps".

A subsequent inspect_analysis_view summary copied that inspection status, reported source_complete: true, and omitted source_coverage. Its unknowns contained only the generic runtime/format-specific-analysis caveat. The FIFO omission sentence remained in limitations. An inventory view of the same retained inspection did return the typed unknown traversal coverage.

Reproduction

Run from the source checkout with its locked dependencies and built runtime available. This reproduction uses the repository's existing MCP harness to launch the actual built server; it does not mock inventory production or construct synthetic Evidence. It reproduced on repeated runs of the FIFO fixture. The final run below also checked the inventory-view workaround.

Create src/domain/repro.test.ts with this body:

import { execFileSync } from "node:child_process";
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { connectLocalToolsMcp } from "../../tests/fixtures/localToolsMcp.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";

it("reports inspection coverage for an inventory that omitted a FIFO", async () => {
  const root = await createTestTempDirectory("rea-coverage-repro-");
  const target = join(root, "fixture.app");
  await mkdir(target);
  await writeFile(join(target, "Info.plist"), "<plist><dict><key>CFBundleExecutable</key><string>main.js</string></dict></plist>");
  await writeFile(join(target, "main.js"), "export const value = 1;");
  execFileSync("mkfifo", [join(target, "omitted.fifo")]);
  const { call } = await connectLocalToolsMcp({
    HOME: process.env.HOME!,
    XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME!,
    TMPDIR: process.env.TMPDIR!,
  });
  const opened = await call("open_binary", { path: target });
  expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
  const complete = await call("inspect_artifact", {});
  expect(complete.isError, JSON.stringify(complete)).not.toBe(true);
  const parent = complete.structuredContent as any;
  const inspection = parent.normalized_result;
  const inventory = inspection.substeps[0].evidence.normalized_result;
  const projected = await call("inspect_analysis_view", {
    source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
    view: { kind: "summary" },
  });
  expect(projected.isError, JSON.stringify(projected)).not.toBe(true);
  const view = (projected.structuredContent as any).normalized_result;
  const inventoryView = await call("inspect_analysis_view", {
    source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
    view: { kind: "inventory", collection: "occurrences", offset: 0, limit: 64 },
  });
  expect(inventoryView.isError, JSON.stringify(inventoryView)).not.toBe(true);
  const inventoryPage = (inventoryView.structuredContent as any).normalized_result;
  expect(inventoryPage.source_coverage).toEqual(inventory.coverage);
  console.log(JSON.stringify({
    inventory_coverage: inventory.coverage,
    inventory_view_source_coverage: inventoryPage.source_coverage,
    inspection_status: inspection.coverage.status,
    summary_inspection_status: view.summary.inspection.coverage.status,
    summary_source_complete: view.summary.source_complete,
    summary_has_source_coverage: Object.hasOwn(view.summary, "source_coverage"),
    summary_unknowns: view.unknowns,
    traversal_limitations: inventory.limitations.filter((line: string) => line.includes("fifo")),
  }, null, 2));
  expect(inventory.coverage).toEqual({ scope: "container-entries", status: "unknown" });
  expect(inspection.coverage.status).toBe("complete-within-substeps");
  expect(view.summary.inspection.coverage.status).toBe("complete-within-substeps");
  expect(view.summary.source_complete).toBe(true);
  expect(Object.hasOwn(view.summary, "source_coverage")).toBe(false);
}, 30_000);

Run the focused source test with an isolated home/config/temp environment:

repro_root=$(mktemp -d)
mkdir -p "$repro_root/home" "$repro_root/config" "$repro_root/tmp"
HOME="$repro_root/home" XDG_CONFIG_HOME="$repro_root/config" TMPDIR="$repro_root/tmp" npm run test:local -- src/domain/repro.test.ts
rm src/domain/repro.test.ts
rm -r "$repro_root"

The assertions document the observed faulty result; they are not assertions of the desired behavior. The test file was deleted after verification.

Expected behavior

The inspection and its summary should preserve the inventory's explicit unknown traversal status as machine-readable coverage. The existing partial inspection status could express this gap; alternatively, a separately exposed source coverage field must make the distinction explicit. A completed inventory substep does not establish exhaustive traversal.

The tool contract requires coverage inline, treats missing coverage as unknown, and preserves partial provider facts. The completeness rule derives completeness from examined records and exhaustiveness rather than diagnostic prose. These requirements apply to the explicit traversal gap here, independently of the format-specific analysis that inspection intentionally leaves unexplored.

source_complete is documented as complete graph-record checks; its true value is not itself a traversal claim. The summary contract distinguishes those counts from source absence.

Impact and scope

P2: misleading machine-readable inspection coverage for a bounded class of successful inventories. The executed fixture had unknown traversal because a FIFO was omitted. The inspection producer's coverage helper ignores the inventory, so this is not specific to the FIFO wording. Other formats and platforms were not exercised.

The complete nested inventory and human-readable omission limitation survive. A practical workaround was verified: request an inventory view and read source_coverage, or inspect the retained nested inventory. No false downstream absence conclusion, data corruption, or security boundary crossing was observed. This does not claim every inspection is incomplete or that all declared format-specific follow-ups must run.

REA version

rea-agents 6.7.0; source baseline a5f982de8d

Runtime and platform

Node.js v26.12.0; Darwin arm64; built production MCP server launched through the repository harness; no deep analysis provider needed.

Target and artifact context

Local fixture.app directory with Info.plist, main.js and an omitted FIFO named omitted.fifo; container-entry traversal coverage unknown.

Interface

MCP

Earliest failure stage

Artifact or evidence registration

Relevant output or evidence

Actually run: npm run test:local -- src/domain/repro.test.ts, with isolated environment as described above. Exit code 0; one test passed. The test printed:

{
  "inventory_coverage": {
    "scope": "container-entries",
    "status": "unknown"
  },
  "inventory_view_source_coverage": {
    "scope": "container-entries",
    "status": "unknown"
  },
  "inspection_status": "complete-within-substeps",
  "summary_inspection_status": "complete-within-substeps",
  "summary_source_complete": true,
  "summary_has_source_coverage": false,
  "summary_unknowns": [
    "Inventory does not establish application runtime behavior or complete format-specific analysis."
  ],
  "traversal_limitations": [
    "Artifact traversal omitted fifo at \"omitted.fifo\"."
  ]
}

Source trace checked at the baseline:

The inventory owner marks traversal unknown when it omits entries, and inventoryCoverage preserves that status with container-entry scope.
ArtifactProvider.inspectArtifact constructs inventory Evidence and passes it to the inspection producer after the supported MCP request is admitted.
The inspection producer fixes omission counts at zero; its coverage helper ignores the inventory and returns complete-within-substeps. At this revision neither partial nor truncated can be emitted by this producer.
Summary source-coverage selection excludes directory summaries. The traversal-unknown message is gated by the same selection. The summary projection copies inspection coverage alongside graph completeness.

Typed outcome or recovery expected

Preserve the inventory's unknown traversal status in the inspection and summary without requiring callers to infer it from limitation prose or fetch another view. Keep the retained inventory Evidence and the distinction between graph-record completeness, traversal coverage, and format-specific semantic analysis.

Analysis or suspected cause

Source-derived: the inspection coverage calculation uses only hard-coded omission counts and never consults the inventory's coverage. The summary then projects this status while excluding the inventory's typed traversal coverage for this root format. This explains the observed mismatch; no historical regression or prior-release behavior was investigated.

Additional context

Read-only searches of open and closed issues and PRs found no same-failure report. The closest closed report, the ZIP non-entry coverage issue, concerns missing limitations for bytes outside ZIP entries. This reproduction instead starts with an explicit unknown traversal status that the inventory already produced and retained.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5837 Originally assigned to: @kaoru0822-kitauji on GitHub. ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Executed through the production MCP server with a local application directory containing ordinary files and a FIFO. After `open_binary`, `inspect_artifact` succeeded. Its nested inventory reported `coverage: {scope: "container-entries", status: "unknown"}` and named the omitted FIFO. The enclosing inspection nevertheless reported `coverage.status: "complete-within-substeps"`. A subsequent `inspect_analysis_view` summary copied that inspection status, reported `source_complete: true`, and omitted `source_coverage`. Its `unknowns` contained only the generic runtime/format-specific-analysis caveat. The FIFO omission sentence remained in limitations. An inventory view of the same retained inspection did return the typed unknown traversal coverage. ### Reproduction Run from the source checkout with its locked dependencies and built runtime available. This reproduction uses the repository's existing MCP harness to launch the actual built server; it does not mock inventory production or construct synthetic Evidence. It reproduced on repeated runs of the FIFO fixture. The final run below also checked the inventory-view workaround. Create `src/domain/repro.test.ts` with this body: ```ts import { execFileSync } from "node:child_process"; import { mkdir, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { expect, it } from "vitest"; import { connectLocalToolsMcp } from "../../tests/fixtures/localToolsMcp.js"; import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js"; it("reports inspection coverage for an inventory that omitted a FIFO", async () => { const root = await createTestTempDirectory("rea-coverage-repro-"); const target = join(root, "fixture.app"); await mkdir(target); await writeFile(join(target, "Info.plist"), "<plist><dict><key>CFBundleExecutable</key><string>main.js</string></dict></plist>"); await writeFile(join(target, "main.js"), "export const value = 1;"); execFileSync("mkfifo", [join(target, "omitted.fifo")]); const { call } = await connectLocalToolsMcp({ HOME: process.env.HOME!, XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME!, TMPDIR: process.env.TMPDIR!, }); const opened = await call("open_binary", { path: target }); expect(opened.isError, JSON.stringify(opened)).not.toBe(true); const complete = await call("inspect_artifact", {}); expect(complete.isError, JSON.stringify(complete)).not.toBe(true); const parent = complete.structuredContent as any; const inspection = parent.normalized_result; const inventory = inspection.substeps[0].evidence.normalized_result; const projected = await call("inspect_analysis_view", { source: { kind: "retained-evidence", evidence_id: parent.evidence_id }, view: { kind: "summary" }, }); expect(projected.isError, JSON.stringify(projected)).not.toBe(true); const view = (projected.structuredContent as any).normalized_result; const inventoryView = await call("inspect_analysis_view", { source: { kind: "retained-evidence", evidence_id: parent.evidence_id }, view: { kind: "inventory", collection: "occurrences", offset: 0, limit: 64 }, }); expect(inventoryView.isError, JSON.stringify(inventoryView)).not.toBe(true); const inventoryPage = (inventoryView.structuredContent as any).normalized_result; expect(inventoryPage.source_coverage).toEqual(inventory.coverage); console.log(JSON.stringify({ inventory_coverage: inventory.coverage, inventory_view_source_coverage: inventoryPage.source_coverage, inspection_status: inspection.coverage.status, summary_inspection_status: view.summary.inspection.coverage.status, summary_source_complete: view.summary.source_complete, summary_has_source_coverage: Object.hasOwn(view.summary, "source_coverage"), summary_unknowns: view.unknowns, traversal_limitations: inventory.limitations.filter((line: string) => line.includes("fifo")), }, null, 2)); expect(inventory.coverage).toEqual({ scope: "container-entries", status: "unknown" }); expect(inspection.coverage.status).toBe("complete-within-substeps"); expect(view.summary.inspection.coverage.status).toBe("complete-within-substeps"); expect(view.summary.source_complete).toBe(true); expect(Object.hasOwn(view.summary, "source_coverage")).toBe(false); }, 30_000); ``` Run the focused source test with an isolated home/config/temp environment: ```sh repro_root=$(mktemp -d) mkdir -p "$repro_root/home" "$repro_root/config" "$repro_root/tmp" HOME="$repro_root/home" XDG_CONFIG_HOME="$repro_root/config" TMPDIR="$repro_root/tmp" npm run test:local -- src/domain/repro.test.ts rm src/domain/repro.test.ts rm -r "$repro_root" ``` The assertions document the observed faulty result; they are not assertions of the desired behavior. The test file was deleted after verification. ### Expected behavior The inspection and its summary should preserve the inventory's explicit unknown traversal status as machine-readable coverage. The existing `partial` inspection status could express this gap; alternatively, a separately exposed source coverage field must make the distinction explicit. A completed inventory substep does not establish exhaustive traversal. [The tool contract](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/tool-design.md#L51-L59) requires coverage inline, treats missing coverage as unknown, and preserves partial provider facts. [The completeness rule](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/tool-design.md#L173-L175) derives completeness from examined records and exhaustiveness rather than diagnostic prose. These requirements apply to the explicit traversal gap here, independently of the format-specific analysis that inspection intentionally leaves unexplored. `source_complete` is documented as complete graph-record checks; its true value is not itself a traversal claim. [The summary contract](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/docs/mcp-contracts.md#L355-L368) distinguishes those counts from source absence. ### Impact and scope P2: misleading machine-readable inspection coverage for a bounded class of successful inventories. The executed fixture had unknown traversal because a FIFO was omitted. The inspection producer's coverage helper ignores the inventory, so this is not specific to the FIFO wording. Other formats and platforms were not exercised. The complete nested inventory and human-readable omission limitation survive. A practical workaround was verified: request an inventory view and read `source_coverage`, or inspect the retained nested inventory. No false downstream absence conclusion, data corruption, or security boundary crossing was observed. This does not claim every inspection is incomplete or that all declared format-specific follow-ups must run. ### REA version rea-agents 6.7.0; source baseline a5f982de8dbe4b092cc20ab89d492b7e7028b758 ### Runtime and platform Node.js v26.12.0; Darwin arm64; built production MCP server launched through the repository harness; no deep analysis provider needed. ### Target and artifact context Local fixture.app directory with Info.plist, main.js and an omitted FIFO named omitted.fifo; container-entry traversal coverage unknown. ### Interface MCP ### Earliest failure stage Artifact or evidence registration ### Relevant output or evidence Actually run: `npm run test:local -- src/domain/repro.test.ts`, with isolated environment as described above. Exit code 0; one test passed. The test printed: ```json { "inventory_coverage": { "scope": "container-entries", "status": "unknown" }, "inventory_view_source_coverage": { "scope": "container-entries", "status": "unknown" }, "inspection_status": "complete-within-substeps", "summary_inspection_status": "complete-within-substeps", "summary_source_complete": true, "summary_has_source_coverage": false, "summary_unknowns": [ "Inventory does not establish application runtime behavior or complete format-specific analysis." ], "traversal_limitations": [ "Artifact traversal omitted fifo at \"omitted.fifo\"." ] } ``` Source trace checked at the baseline: - [The inventory owner](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/artifacts/inventory/scanReader.ts#L123-L138) marks traversal unknown when it omits entries, and [inventoryCoverage](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/artifacts/inventory/reader.ts#L51-L72) preserves that status with container-entry scope. - [ArtifactProvider.inspectArtifact](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/artifacts/ArtifactProvider.ts#L422-L456) constructs inventory Evidence and passes it to the inspection producer after the supported MCP request is admitted. - [The inspection producer](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/artifactInspection.ts#L143-L148) fixes omission counts at zero; [its coverage helper](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/artifactInspection.ts#L430-L436) ignores the inventory and returns `complete-within-substeps`. At this revision neither `partial` nor `truncated` can be emitted by this producer. - [Summary source-coverage selection](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisView/artifactInventoryView.ts#L248-L254) excludes directory summaries. [The traversal-unknown message](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisView/artifactInventoryView.ts#L273-L291) is gated by the same selection. [The summary projection](https://github.com/morluto/rea/blob/a5f982de8dbe4b092cc20ab89d492b7e7028b758/src/domain/analysisView/artifactInventoryView.ts#L331-L342) copies inspection coverage alongside graph completeness. ### Typed outcome or recovery expected Preserve the inventory's unknown traversal status in the inspection and summary without requiring callers to infer it from limitation prose or fetch another view. Keep the retained inventory Evidence and the distinction between graph-record completeness, traversal coverage, and format-specific semantic analysis. ### Analysis or suspected cause Source-derived: the inspection coverage calculation uses only hard-coded omission counts and never consults the inventory's coverage. The summary then projects this status while excluding the inventory's typed traversal coverage for this root format. This explains the observed mismatch; no historical regression or prior-release behavior was investigated. ### Additional context Read-only searches of open and closed issues and PRs found no same-failure report. The closest closed report, [the ZIP non-entry coverage issue](https://github.com/morluto/rea/issues/1485), concerns missing limitations for bytes outside ZIP entries. This reproduction instead starts with an explicit unknown traversal status that the inventory already produced and retained.

bsakaguchi closed this issue

2026-10-31 00:41:35 +00:00

bsakaguchi commented

2026-10-31 00:41:36 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 21, 2026):

Confirmed — the inspection producer's coverage helper ignoring the nested inventory's typed unknown status turns an explicit traversal gap into "complete-within-substeps", which is misleading for machine consumers (and the summary then propagates source_complete: true with source_coverage dropped entirely). The inventory view already exposes the correct typed coverage, so propagating inventory.coverage into the inspection/summary projection — or at minimum demoting the status when traversal is unknown — aligns with the tool contract's "missing coverage = unknown" rule.

@XinwangTao commented on GitHub (Oct 21, 2026): Confirmed — the inspection producer's coverage helper ignoring the nested inventory's typed unknown status turns an explicit traversal gap into "complete-within-substeps", which is misleading for machine consumers (and the summary then propagates source_complete: true with source_coverage dropped entirely). The inventory view already exposes the correct typed coverage, so propagating inventory.coverage into the inspection/summary projection — or at minimum demoting the status when traversal is unknown — aligns with the tool contract's "missing coverage = unknown" rule.

bsakaguchi referenced this issue

2026-10-31 01:09:36 +00:00

[PR #3534] [CLOSED] fix(javascript): keep unresolved network request targets visible as partial endpoints #5810

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3534

No description provided.

Rows
Columns