morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5881] [Bug] Unavailable JSON modules report zero omitted keys instead of unknown #3562

New issue

Closed

opened 2026-10-31 00:41:54 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:41:54 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 21, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5881

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The built analyze-javascript-application CLI returns omitted_json_top_level_keys: 0 for a JSON file whose text cannot be decoded, alongside json_parse_status: "unavailable". The JSON analysis contract represents this omission count as null, so the graph projection changes an unknown count into a numeric zero.

The result still contains a separate unavailability observation and partial graph coverage. The incorrect value is the file observation's numeric omission field.

Reproduction

Executed against the built source checkout:

Create an application directory containing these files:
- package.json: {"name":"json-coverage-repro","version":"1.0.0","main":"main.js"}
- main.js: const config = require("./config.json");
- config.json: exactly one byte, hexadecimal ff.
From the source checkout, run node scripts/rea.mjs analyze-javascript-application <application-directory> --format json.
Read normalized_result.graph.nodes, select the observation whose properties.path is config.json and properties.file_kind is json, and inspect its properties.

The command exited successfully. The single run returned the output below. No mocks, native analysis provider, or target-code execution were needed.

Expected behavior

Preserve the unknown omission count for an unavailable JSON observation, for example omitted_json_top_level_keys: null while json_parse_status remains "unavailable".

The JSON observation type explicitly assigns omitted_top_level_keys: null to the unavailable variant. The JSON producer returns that variant when text is unavailable.

Impact and scope

An unavailable JSON module exposes a misleading numeric coverage fact in returned application Evidence. Consumers reading only the omission count cannot distinguish unknown coverage from a measured zero.

This was observed for one inventoried .json file with invalid UTF-8. The CLI still completes and reports partial graph coverage. Consumers can work around the incorrect count by checking json_parse_status: "unavailable" or text_status: "invalid-utf8" and treating the count as unknown. No downstream incorrect decision or broader runtime impact was tested.

REA version

6.7.0, source revision 52415975942a4b9c53a817915b313ccc404f280e.

Runtime and platform

macOS 27.0.1 arm64, Node.js v26.12.0.

Target and artifact context

Extracted CommonJS application directory with a normal package entrypoint and one .json file containing invalid UTF-8. All files were readable and inventoried.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The actual file observation returned by the CLI included:

{
  "path": "config.json",
  "bytes": 1,
  "file_kind": "json",
  "text_status": "invalid-utf8",
  "parse_status": null,
  "json_parse_status": "unavailable",
  "json_top_level_keys": [],
  "omitted_json_top_level_keys": 0
}

The same result contained a separate observation:

{
  "path": "config.json",
  "operation": "parse-json-module",
  "limitation": "JSON module text was unavailable: invalid-utf8."
}

Graph coverage was {"status":"partial","truncated":false,"omitted_count":null,"limits":[]}. These accompanying facts show that unavailability was recognized before the count was projected.

Typed outcome or recovery expected

Keep an explicit unknown omission count for unavailable JSON modules instead of replacing it with zero.

Analysis or suspected cause

Source-derived: the graph property projection uses json?.omitted_top_level_keys ?? 0. This maps both an absent observation and the unavailable variant's explicit null to zero.

The artifact reader uses fatal UTF-8 decoding and retains an invalid-UTF-8 file with unavailable text. The production analysis then records its JSON observation. This is a reachable input state rather than a manually constructed internal value.

Additional context

No response

Originally created by @coygeek on GitHub (Oct 21, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5881 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The built `analyze-javascript-application` CLI returns `omitted_json_top_level_keys: 0` for a JSON file whose text cannot be decoded, alongside `json_parse_status: "unavailable"`. The JSON analysis contract represents this omission count as `null`, so the graph projection changes an unknown count into a numeric zero. The result still contains a separate unavailability observation and partial graph coverage. The incorrect value is the file observation's numeric omission field. ### Reproduction Executed against the built source checkout: 1. Create an application directory containing these files: - `package.json`: `{"name":"json-coverage-repro","version":"1.0.0","main":"main.js"}` - `main.js`: `const config = require("./config.json");` - `config.json`: exactly one byte, hexadecimal `ff`. 2. From the source checkout, run `node scripts/rea.mjs analyze-javascript-application <application-directory> --format json`. 3. Read `normalized_result.graph.nodes`, select the observation whose `properties.path` is `config.json` and `properties.file_kind` is `json`, and inspect its properties. The command exited successfully. The single run returned the output below. No mocks, native analysis provider, or target-code execution were needed. ### Expected behavior Preserve the unknown omission count for an unavailable JSON observation, for example `omitted_json_top_level_keys: null` while `json_parse_status` remains `"unavailable"`. The [JSON observation type](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/javascript/JavaScriptArtifactAnalysisTypes.ts#L100-L120) explicitly assigns `omitted_top_level_keys: null` to the unavailable variant. The [JSON producer](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/javascript/JavaScriptJsonModules.ts#L5-L17) returns that variant when text is unavailable. ### Impact and scope An unavailable JSON module exposes a misleading numeric coverage fact in returned application Evidence. Consumers reading only the omission count cannot distinguish unknown coverage from a measured zero. This was observed for one inventoried `.json` file with invalid UTF-8. The CLI still completes and reports partial graph coverage. Consumers can work around the incorrect count by checking `json_parse_status: "unavailable"` or `text_status: "invalid-utf8"` and treating the count as unknown. No downstream incorrect decision or broader runtime impact was tested. ### REA version 6.7.0, source revision `52415975942a4b9c53a817915b313ccc404f280e`. ### Runtime and platform macOS 27.0.1 arm64, Node.js v26.12.0. ### Target and artifact context Extracted CommonJS application directory with a normal package entrypoint and one `.json` file containing invalid UTF-8. All files were readable and inventoried. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The actual file observation returned by the CLI included: ```json { "path": "config.json", "bytes": 1, "file_kind": "json", "text_status": "invalid-utf8", "parse_status": null, "json_parse_status": "unavailable", "json_top_level_keys": [], "omitted_json_top_level_keys": 0 } ``` The same result contained a separate observation: ```json { "path": "config.json", "operation": "parse-json-module", "limitation": "JSON module text was unavailable: invalid-utf8." } ``` Graph coverage was `{"status":"partial","truncated":false,"omitted_count":null,"limits":[]}`. These accompanying facts show that unavailability was recognized before the count was projected. ### Typed outcome or recovery expected Keep an explicit unknown omission count for unavailable JSON modules instead of replacing it with zero. ### Analysis or suspected cause Source-derived: the [graph property projection](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/javascript/JavaScriptArtifactGraphStructure.ts#L604-L616) uses `json?.omitted_top_level_keys ?? 0`. This maps both an absent observation and the unavailable variant's explicit `null` to zero. The [artifact reader](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/artifacts/javascript/JavaScriptArtifactFiles.ts#L349-L373) uses fatal UTF-8 decoding and retains an invalid-UTF-8 file with unavailable text. The [production analysis](https://github.com/morluto/rea/blob/52415975942a4b9c53a817915b313ccc404f280e/src/application/javascript/JavaScriptArtifactAnalysis.ts#L484-L493) then records its JSON observation. This is a reachable input state rather than a manually constructed internal value. ### Additional context _No response_

bsakaguchi closed this issue

2026-10-31 00:41:54 +00:00

bsakaguchi referenced this issue

2026-10-31 01:09:44 +00:00

[PR #3562] [CLOSED] fix(reference): preserve physical symlink targets #5827

bsakaguchi referenced this issue

2026-11-01 21:11:07 +00:00

[GH-ISSUE #5654] [Tracker] Open PRs that can't merge yet: conflicts and red CI (October 2026) #3425