morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5904] [Bug] Semantic graph validation admits contradictory truncation coverage #3582

New issue

Closed

opened 2026-10-31 00:42:05 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:42:05 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5904

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The semantic graph integrity boundary accepts a graph that declares truncation while claiming zero omitted nodes, zero omitted relations, and no omitted unknown occurrences. It also accepts status: "unknown" or "unavailable" with truncated: true.

This is reachable through public CLI workflows. Starting with analysis of a valid source file, changing only semantic coverage, and recomputing the graph and Evidence identifiers produces inline Evidence accepted by both trace-javascript-semantics and inspect-analysis-view. The summary preserves the contradictory coverage fields. For the unavailable variant, semantic tracing returns status: "unsupported" and unavailable coverage even though the same retained module seed returns found under the partial variant.

Reproduction

From a built checkout at the revision below, set REPRO_DIR to an empty absolute working directory and run this shell block from the repository root. This block was executed locally. It analyzes inert source with the production worker, constructs serialized Evidence requests using the production factories, and invokes both public CLI commands. The target JavaScript is never executed.

export REPRO_DIR
mkdir -p "$REPRO_DIR/app"
printf '%s\n' 'export const answer = 42;' > "$REPRO_DIR/app/valid.js"
node scripts/rea.mjs analyze-javascript-application "$REPRO_DIR/app" \
  --format json > "$REPRO_DIR/application.json"
node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import fs from 'node:fs';
import { spawnSync } from 'node:child_process';
import { createEvidence } from './dist/domain/evidence.js';
import {
  createJavaScriptSemanticGraph,
  javaScriptSemanticGraphSchema,
} from './dist/domain/javascript/javascriptSemanticGraph.js';
import {
  javascriptApplicationAnalysisResultSchema,
} from './dist/domain/javascript/javascriptApplicationAnalysis.js';

const dir = process.env.REPRO_DIR;
const source = JSON.parse(fs.readFileSync(`${dir}/application.json`));
assert.equal(source.normalized_result.statistics.parse_failures, 0);
for (const status of ['unknown', 'unavailable', 'partial']) {
  const result = structuredClone(source.normalized_result);
  const { graph_id, ...input } = result.semantic_graph;
  input.coverage = {
    ...input.coverage, status, truncated: true,
    omitted_nodes: 0, omitted_relations: 0,
    limits: [{ name: 'semantic_graph_node_ceiling', value: 1, unit: 'items' }],
  };
  result.semantic_graph = createJavaScriptSemanticGraph(input);
  assert.equal(javaScriptSemanticGraphSchema.safeParse(
    JSON.parse(JSON.stringify(result.semantic_graph))).success, true);
  assert.equal(javascriptApplicationAnalysisResultSchema.safeParse(result).success, true);
  const subject = source.subject;
  const evidence = createEvidence({
    path: subject.local_path, sha256: subject.digest.sha256, format: subject.format,
  }, source.provider, {
    predicateType: source.predicate_type, operation: source.operation,
    parameters: source.parameters, result,
    confidence: source.confidence, authority: source.authority,
  });
  const requests = {
    'trace-javascript-semantics': {
      application: evidence,
      query: {
        seed: { kind: 'semantic-node', node_id: result.semantic_graph.root_node_ids[0] },
        direction: 'forward-influence',
      },
    },
    'inspect-analysis-view': {
      source: { kind: 'inline', evidence }, view: { kind: 'summary' },
    },
  };
  for (const [command, request] of Object.entries(requests)) {
    const file = `${dir}/${status}-${command}-request.json`;
    fs.writeFileSync(file, JSON.stringify(request));
    const run = spawnSync(process.execPath,
      ['scripts/rea.mjs', command, file, '--format', 'json'], { encoding: 'utf8' });
    assert.equal(run.status, 0, run.stderr);
    const reply = JSON.parse(run.stdout);
    assert.equal(reply.error, undefined, JSON.stringify(reply));
    fs.writeFileSync(`${dir}/${status}-${command}-result.json`, run.stdout);
    const r = reply.normalized_result;
    console.log(JSON.stringify(command === 'inspect-analysis-view'
      ? { input_status: status, command, semantic: {
          status: r.summary.coverage.semantic.status,
          truncated: r.summary.coverage.semantic.truncated,
          omitted_nodes: r.summary.coverage.semantic.omitted_nodes,
          omitted_relations: r.summary.coverage.semantic.omitted_relations,
        } }
      : { input_status: status, command, status: r.status, coverage: r.coverage }));
  }
}
JS

The assertions describe the current incorrect acceptance. Each of the three coverage variants was admitted by graph construction, serialized graph validation, application-result validation, and both public workflows.

Expected behavior

Truncated semantic coverage must identify an effective bound, use partial status, and acknowledge omitted work. A graph that claims zero omissions of every supported kind must not also declare truncation. Reject the inconsistent input at graph/Evidence validation with a useful coverage diagnostic.

The coverage invariant states the truncation/status/omission coupling for application graph evidence. The companion semantic graph producer independently follows the same policy: node truncation yields partial status and unknown node/relation omission counts. Its separate omitted-unknown accounting permits zero node/relation omissions when dropped unknown occurrences explain truncation.

Impact and scope

P2: bounded coverage-integrity defect in imported inline Evidence. Consumers receive internally inconsistent omission and availability claims through a validation boundary. No lost source facts, security effect, or instance of the normal analyzer producing these invalid fields was observed.

A demonstrated workaround for callers constructing Evidence is partial coverage with null omitted node/relation extents and a named limit; that representation validates and both public workflows succeed. This does not repair admission of inconsistent incoming records.

REA version

rea-agents 6.7.0; source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64; Node.js v26.12.0. Used the built CLI and production static JavaScript worker, with no external analysis provider.

Target and artifact context

A local directory containing only valid.js with export const answer = 42;. The initial analysis reports zero parse failures, four semantic nodes, two data-flow relations, and no semantic unknowns. The changed records keep source identity, structural graph, semantic facts and family coverage intact, while deriving fresh graph and outer Evidence commitments.

Interface

CLI

Earliest failure stage

Request parsing or validation

Relevant output or evidence

Representative observed output from the reproduction:

{"input_status":"unknown","command":"trace-javascript-semantics","status":"found","coverage":{"status":"partial"}}
{"input_status":"unknown","command":"inspect-analysis-view","semantic":{"status":"unknown","truncated":true,"omitted_nodes":0,"omitted_relations":0}}
{"input_status":"unavailable","command":"trace-javascript-semantics","status":"unsupported","coverage":{"status":"unavailable"}}
{"input_status":"unavailable","command":"inspect-analysis-view","semantic":{"status":"unavailable","truncated":true,"omitted_nodes":0,"omitted_relations":0}}
{"input_status":"partial","command":"trace-javascript-semantics","status":"found","coverage":{"status":"partial"}}
{"input_status":"partial","command":"inspect-analysis-view","semantic":{"status":"partial","truncated":true,"omitted_nodes":0,"omitted_relations":0}}

Additional executed controls isolated the missing checks:

The unmodified analysis validates.
Truncated unknown coverage with null omitted extents is also accepted, exposing the status-coupling gap independently of zero counts.
Truncated partial coverage with null omitted extents and a named limit is accepted.
Truncated partial coverage without a limit is rejected: Truncated graph coverage must identify an effective limit.
Complete coverage with truncation is rejected: Complete graph coverage cannot omit or truncate semantic facts.

These are local runtime observations. Current upstream main was fetched and checked; the relevant validator, schema, intake and projection files have no changes since the tested revision. No runtime execution of a newer revision is claimed.

Typed outcome or recovery expected

Return an invalid-request coverage diagnostic for inconsistent imported graphs. Preserve valid partial graphs with unknown omission extents, and preserve the legitimate case where positive omitted unknown occurrences explain truncation despite zero omitted node/relation counts. Apply the same invariant through graph creation and serialized Evidence intake.

Analysis or suspected cause

Source-derived: checkCoverageSteps checks truncation/status coupling only when omitted unknown occurrences are present. Its general truncation check requires a limit, while its complete-status branch rejects truncation. Neither branch rejects truncated unknown/unavailable coverage or all-zero omissions when omitted-unknown accounting is absent.

Public application Evidence intake validates the analysis result and then trusts its semantic graph. Trace assessment uses the admitted availability status, while summary coverage projection retains the stored fields.

Additional context

Related report #2576 concerned complete coverage with a retained unresolved frontier; that check is present and is a different invariant. The unavailable-source handling in open PR #2092 addresses parse-failure projection, while its current validator diff does not add these truncation checks.

NOT_ENOUGH_INFO: the introducing revision and a last known good version have not been established. MCP execution was not exercised; the public CLI route was.

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5904 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The semantic graph integrity boundary accepts a graph that declares truncation while claiming zero omitted nodes, zero omitted relations, and no omitted unknown occurrences. It also accepts `status: "unknown"` or `"unavailable"` with `truncated: true`. This is reachable through public CLI workflows. Starting with analysis of a valid source file, changing only semantic coverage, and recomputing the graph and Evidence identifiers produces inline Evidence accepted by both `trace-javascript-semantics` and `inspect-analysis-view`. The summary preserves the contradictory coverage fields. For the `unavailable` variant, semantic tracing returns `status: "unsupported"` and unavailable coverage even though the same retained module seed returns `found` under the partial variant. ### Reproduction From a built checkout at the revision below, set `REPRO_DIR` to an empty absolute working directory and run this shell block from the repository root. This block was executed locally. It analyzes inert source with the production worker, constructs serialized Evidence requests using the production factories, and invokes both public CLI commands. The target JavaScript is never executed. ```sh export REPRO_DIR mkdir -p "$REPRO_DIR/app" printf '%s\n' 'export const answer = 42;' > "$REPRO_DIR/app/valid.js" node scripts/rea.mjs analyze-javascript-application "$REPRO_DIR/app" \ --format json > "$REPRO_DIR/application.json" node --input-type=module <<'JS' import assert from 'node:assert/strict'; import fs from 'node:fs'; import { spawnSync } from 'node:child_process'; import { createEvidence } from './dist/domain/evidence.js'; import { createJavaScriptSemanticGraph, javaScriptSemanticGraphSchema, } from './dist/domain/javascript/javascriptSemanticGraph.js'; import { javascriptApplicationAnalysisResultSchema, } from './dist/domain/javascript/javascriptApplicationAnalysis.js'; const dir = process.env.REPRO_DIR; const source = JSON.parse(fs.readFileSync(`${dir}/application.json`)); assert.equal(source.normalized_result.statistics.parse_failures, 0); for (const status of ['unknown', 'unavailable', 'partial']) { const result = structuredClone(source.normalized_result); const { graph_id, ...input } = result.semantic_graph; input.coverage = { ...input.coverage, status, truncated: true, omitted_nodes: 0, omitted_relations: 0, limits: [{ name: 'semantic_graph_node_ceiling', value: 1, unit: 'items' }], }; result.semantic_graph = createJavaScriptSemanticGraph(input); assert.equal(javaScriptSemanticGraphSchema.safeParse( JSON.parse(JSON.stringify(result.semantic_graph))).success, true); assert.equal(javascriptApplicationAnalysisResultSchema.safeParse(result).success, true); const subject = source.subject; const evidence = createEvidence({ path: subject.local_path, sha256: subject.digest.sha256, format: subject.format, }, source.provider, { predicateType: source.predicate_type, operation: source.operation, parameters: source.parameters, result, confidence: source.confidence, authority: source.authority, }); const requests = { 'trace-javascript-semantics': { application: evidence, query: { seed: { kind: 'semantic-node', node_id: result.semantic_graph.root_node_ids[0] }, direction: 'forward-influence', }, }, 'inspect-analysis-view': { source: { kind: 'inline', evidence }, view: { kind: 'summary' }, }, }; for (const [command, request] of Object.entries(requests)) { const file = `${dir}/${status}-${command}-request.json`; fs.writeFileSync(file, JSON.stringify(request)); const run = spawnSync(process.execPath, ['scripts/rea.mjs', command, file, '--format', 'json'], { encoding: 'utf8' }); assert.equal(run.status, 0, run.stderr); const reply = JSON.parse(run.stdout); assert.equal(reply.error, undefined, JSON.stringify(reply)); fs.writeFileSync(`${dir}/${status}-${command}-result.json`, run.stdout); const r = reply.normalized_result; console.log(JSON.stringify(command === 'inspect-analysis-view' ? { input_status: status, command, semantic: { status: r.summary.coverage.semantic.status, truncated: r.summary.coverage.semantic.truncated, omitted_nodes: r.summary.coverage.semantic.omitted_nodes, omitted_relations: r.summary.coverage.semantic.omitted_relations, } } : { input_status: status, command, status: r.status, coverage: r.coverage })); } } JS ``` The assertions describe the current incorrect acceptance. Each of the three coverage variants was admitted by graph construction, serialized graph validation, application-result validation, and both public workflows. ### Expected behavior Truncated semantic coverage must identify an effective bound, use partial status, and acknowledge omitted work. A graph that claims zero omissions of every supported kind must not also declare truncation. Reject the inconsistent input at graph/Evidence validation with a useful coverage diagnostic. The [coverage invariant](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/docs/javascript-application-graph.md#L151-L159) states the truncation/status/omission coupling for application graph evidence. The companion [semantic graph producer](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/javascript/JavaScriptSemanticGraphBuilder.ts#L280-L347) independently follows the same policy: node truncation yields partial status and unknown node/relation omission counts. Its separate omitted-unknown accounting permits zero node/relation omissions when dropped unknown occurrences explain truncation. ### Impact and scope P2: bounded coverage-integrity defect in imported inline Evidence. Consumers receive internally inconsistent omission and availability claims through a validation boundary. No lost source facts, security effect, or instance of the normal analyzer producing these invalid fields was observed. A demonstrated workaround for callers constructing Evidence is partial coverage with null omitted node/relation extents and a named limit; that representation validates and both public workflows succeed. This does not repair admission of inconsistent incoming records. ### REA version rea-agents 6.7.0; source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64; Node.js v26.12.0. Used the built CLI and production static JavaScript worker, with no external analysis provider. ### Target and artifact context A local directory containing only `valid.js` with `export const answer = 42;`. The initial analysis reports zero parse failures, four semantic nodes, two data-flow relations, and no semantic unknowns. The changed records keep source identity, structural graph, semantic facts and family coverage intact, while deriving fresh graph and outer Evidence commitments. ### Interface CLI ### Earliest failure stage Request parsing or validation ### Relevant output or evidence Representative observed output from the reproduction: ```json {"input_status":"unknown","command":"trace-javascript-semantics","status":"found","coverage":{"status":"partial"}} {"input_status":"unknown","command":"inspect-analysis-view","semantic":{"status":"unknown","truncated":true,"omitted_nodes":0,"omitted_relations":0}} {"input_status":"unavailable","command":"trace-javascript-semantics","status":"unsupported","coverage":{"status":"unavailable"}} {"input_status":"unavailable","command":"inspect-analysis-view","semantic":{"status":"unavailable","truncated":true,"omitted_nodes":0,"omitted_relations":0}} {"input_status":"partial","command":"trace-javascript-semantics","status":"found","coverage":{"status":"partial"}} {"input_status":"partial","command":"inspect-analysis-view","semantic":{"status":"partial","truncated":true,"omitted_nodes":0,"omitted_relations":0}} ``` Additional executed controls isolated the missing checks: - The unmodified analysis validates. - Truncated `unknown` coverage with null omitted extents is also accepted, exposing the status-coupling gap independently of zero counts. - Truncated partial coverage with null omitted extents and a named limit is accepted. - Truncated partial coverage without a limit is rejected: `Truncated graph coverage must identify an effective limit`. - Complete coverage with truncation is rejected: `Complete graph coverage cannot omit or truncate semantic facts`. These are local runtime observations. Current upstream `main` was fetched and checked; the relevant validator, schema, intake and projection files have no changes since the tested revision. No runtime execution of a newer revision is claimed. ### Typed outcome or recovery expected Return an invalid-request coverage diagnostic for inconsistent imported graphs. Preserve valid partial graphs with unknown omission extents, and preserve the legitimate case where positive omitted unknown occurrences explain truncation despite zero omitted node/relation counts. Apply the same invariant through graph creation and serialized Evidence intake. ### Analysis or suspected cause Source-derived: [checkCoverageSteps](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/javascript/javascriptSemanticGraph.ts#L265-L315) checks truncation/status coupling only when omitted unknown occurrences are present. Its general truncation check requires a limit, while its complete-status branch rejects truncation. Neither branch rejects truncated unknown/unavailable coverage or all-zero omissions when omitted-unknown accounting is absent. [Public application Evidence intake](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/javascript/JavaScriptApplicationEvidenceGraph.ts#L122-L131) validates the analysis result and then trusts its semantic graph. [Trace assessment](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/javascript/javascriptSemanticQueryAssessment.ts#L32-L63) uses the admitted availability status, while [summary coverage projection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/domain/analysisView/javascriptApplicationCoverage.ts#L22-L43) retains the stored fields. ### Additional context Related report #2576 concerned complete coverage with a retained unresolved frontier; that check is present and is a different invariant. The unavailable-source handling in open PR #2092 addresses parse-failure projection, while its current validator diff does not add these truncation checks. NOT_ENOUGH_INFO: the introducing revision and a last known good version have not been established. MCP execution was not exercised; the public CLI route was.

bsakaguchi closed this issue

2026-10-31 00:42:05 +00:00

bsakaguchi referenced this issue

2026-11-01 21:10:53 +00:00

[GH-ISSUE #3582] [Tracker] Ready to close or merge: issues and pull requests (October 2026) #1330

bsakaguchi referenced this issue

2026-11-01 21:10:54 +00:00

[GH-ISSUE #3795] [Tracker] Open documentation issues and pull requests (October 2026) #1468

bsakaguchi referenced this issue

2026-11-01 21:10:58 +00:00

[GH-ISSUE #3796] [Tracker] Open feature requests by area (October 2026) #1471

bsakaguchi referenced this issue