morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3869] [Bug] Ghidra doctor passes version check for malformed continued properties #1489

New issue

Closed

opened 2026-10-16 21:06:08 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:06:08 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3869

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Provider or capability discovery

Observed behavior

Running the Ghidra provider doctor against an application.properties whose logical property is formed by a backslash continuation into a line beginning with a # or ! comment marker leaves the provider version populated and the version check passing, even though the Java Properties.load contract treats that joined line as property data and the malformed \uZZZZ escape in it must invalidate the file. This repeats the reproduction of closed #3179 on the tested build, so it is best handled as a reopen or follow-up rather than a new defect.

For both continued_hash and continued_bang, doctor retains providerVersion: "12.1.4" and reports version.ok: true; the malformed continuation line silently disappears during parsing. The escaped_hash control, carrying the same malformed escape outside a continuation, correctly fails with version_unresolved. All five fixtures exit 1 only because they omit other installation prerequisites, so this concerns the incorrect version check and retained metadata, not full-provider admission.

Reproduction

Run from a source checkout root; the probe invokes scripts/rea.mjs directly, creates isolated metadata and HOME/XDG directories, calls the real doctor CLI for five metadata fixtures, and removes its fixtures afterward. No Java or Ghidra installation is required.

nice -n 10 node --input-type=module <<'JS'
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
import { join } from 'node:path';
import { tmpdir } from 'node:os';
import { spawnSync } from 'node:child_process';
const root = await mkdtemp(join(tmpdir(), 'rea-ghidra-properties-'));
try {
  const home = join(root, 'home');
  await mkdir(home);
  const prefix = 'application.version=12.1.4\napplication.java.min=21\n';
  const cases = {
    continued_hash: '\\\n#unused=\\uZZZZ\n',
    continued_bang: '\\\n!unused=\\uZZZZ\n',
    escaped_hash: '\\#unused=\\uZZZZ\n',
    real_comment: '#unused=\\uZZZZ\n',
    corrected: '\\\n#unused=\\u0041\n',
  };
  for (const [name, suffix] of Object.entries(cases)) {
    const install = join(root, name);
    await mkdir(join(install, 'Ghidra'), { recursive: true });
    await writeFile(join(install, 'Ghidra/application.properties'), prefix + suffix, 'latin1');
    const run = spawnSync(process.execPath,
      ['scripts/rea.mjs', 'doctor', '--provider', 'ghidra', '--format', 'json'], {
        encoding: 'utf8', timeout: 20000,
        env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, 'config'),
          XDG_CACHE_HOME: join(home, 'cache'), XDG_DATA_HOME: join(home, 'data'),
          GHIDRA_INSTALL_DIR: install },
      });
    if (run.error) throw run.error;
    const ghidra = JSON.parse(run.stdout).providerInspections.find(p => p.id === 'ghidra');
    const version = ghidra.checks.find(c => c.name === 'version');
    console.log(JSON.stringify({ name, exit: run.status, providerVersion: ghidra.providerVersion,
      version: { ok: version.ok, code: version.code, detail: version.detail } }));
  }
} finally { await rm(root, { recursive: true, force: true }); }
JS

Observed output, one JSON line per fixture:

{"name":"continued_hash","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}}
{"name":"continued_bang","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}}
{"name":"escaped_hash","exit":1,"providerVersion":null,"version":{"ok":false,"code":"version_unresolved","detail":"application.properties could not be parsed: Malformed Unicode escape in Java properties file"}}
{"name":"real_comment","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}}
{"name":"corrected","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}}

Expected behavior

Doctor should report providerVersion: null and a failed version check with code version_unresolved for both the continued-# and continued-! fixtures, exactly as it already does for the escaped_hash control, because the joined logical property contains a malformed Unicode escape that the documented parser contract must reject.

Impact and scope

Doctor falsely reports a passing version check and retains a provider version for a caller-managed application.properties that the documented parser contract requires it to reject; the failure is bounded to this continuation-into-comment-marker form, with no data loss or provider admission demonstrated.

Workaround: available. A well-formed application.properties without a comment-marker continuation is validated correctly, as the escaped_hash, corrected, and real_comment controls show; correcting the malformed escape to \u0041 preserves the property and restores correct validation.

Reporter's impact estimate: P2 (supported).

REA version

6.5.0, built from source at main commit 034342ba28 (latest npm release is 6.5.0)

Runtime and platform

Node.js v26.10.0; macOS 27.0.1 arm64

Target and artifact context

Ghidra application.properties fixture (metadata only)

Interface

CLI

Earliest failure stage

Setup or provider admission

Relevant output or evidence

Expected source: src/ghidra/GhidraJavaProperties.ts:3 documents Java Properties.load(InputStream) semantics, under which a # or ! that begins continuation data is property data rather than a physical comment; src/ghidra/GhidraInstallationPolicy.ts:54 requires a malformed escape to invalidate the whole file, and lines 64-70 clear the provider version when parsing fails.
Failure source: the observed output above, with src/ghidra/GhidraJavaProperties.ts:60 treating the marker-bearing physical line as continuation and line 73 joining it, after which the second comment check at line 11 drops the whole logical property before escape decoding, and src/ghidra/GhidraInstallation.ts:784 passes the retained release into the version check (loaded via src/cli/setupCommands.ts:124, src/doctorRuntime.ts:44, src/ghidra/GhidraDoctor.ts:62, and src/ghidra/GhidraInstallation.ts:423 and 481).
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete

Typed outcome or recovery expected

Doctor should have reported providerVersion: null and a failed version check with code version_unresolved, as it already does for the escaped-key control, instead of retaining 12.1.4 and reporting version.ok: true.

Restoration check:

Failing observable: the continued_hash and continued_bang fixtures yield providerVersion: "12.1.4" and version.ok: true. Passing observable: the same fixtures yield providerVersion: null and a failed version check with code version_unresolved, matching the escaped_hash control. Control: the corrected fixture (escape \u0041) and the real_comment fixture keep passing, confirming that well-formed inputs and genuine comments are unaffected.

Analysis or suspected cause

src/ghidra/GhidraJavaProperties.ts:60 and :73 join the comment-marker continuation line as continuation data, then the second comment check at :11 drops the whole logical property before escape decoding, so the malformed escape is never seen; src/ghidra/GhidraInstallation.ts:784 then passes the retained release into the version check. Reproduction path: src/cli/setupCommands.ts:124 -> src/doctorRuntime.ts:44 -> src/ghidra/GhidraDoctor.ts:62 -> src/ghidra/GhidraInstallation.ts:423 and :481.

Additional context

The audit tracker #3537 indexes closed #3179 but does not separately describe this defect, and closed #2785 concerns JVM architecture admission and is distinct.

This report came from AI-assisted source review with local reproduction where stated above; the evidence labels distinguish observed runs from source-only reasoning.

Originally created by @coygeek on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3869 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Provider or capability discovery ### Observed behavior Running the Ghidra provider doctor against an `application.properties` whose logical property is formed by a backslash continuation into a line beginning with a `#` or `!` comment marker leaves the provider version populated and the `version` check passing, even though the Java `Properties.load` contract treats that joined line as property data and the malformed `\uZZZZ` escape in it must invalidate the file. This repeats the reproduction of closed #3179 on the tested build, so it is best handled as a reopen or follow-up rather than a new defect. For both `continued_hash` and `continued_bang`, doctor retains `providerVersion: "12.1.4"` and reports `version.ok: true`; the malformed continuation line silently disappears during parsing. The `escaped_hash` control, carrying the same malformed escape outside a continuation, correctly fails with `version_unresolved`. All five fixtures exit 1 only because they omit other installation prerequisites, so this concerns the incorrect version check and retained metadata, not full-provider admission. ### Reproduction Run from a source checkout root; the probe invokes `scripts/rea.mjs` directly, creates isolated metadata and HOME/XDG directories, calls the real doctor CLI for five metadata fixtures, and removes its fixtures afterward. No Java or Ghidra installation is required. ```sh nice -n 10 node --input-type=module <<'JS' import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; import { spawnSync } from 'node:child_process'; const root = await mkdtemp(join(tmpdir(), 'rea-ghidra-properties-')); try { const home = join(root, 'home'); await mkdir(home); const prefix = 'application.version=12.1.4\napplication.java.min=21\n'; const cases = { continued_hash: '\\\n#unused=\\uZZZZ\n', continued_bang: '\\\n!unused=\\uZZZZ\n', escaped_hash: '\\#unused=\\uZZZZ\n', real_comment: '#unused=\\uZZZZ\n', corrected: '\\\n#unused=\\u0041\n', }; for (const [name, suffix] of Object.entries(cases)) { const install = join(root, name); await mkdir(join(install, 'Ghidra'), { recursive: true }); await writeFile(join(install, 'Ghidra/application.properties'), prefix + suffix, 'latin1'); const run = spawnSync(process.execPath, ['scripts/rea.mjs', 'doctor', '--provider', 'ghidra', '--format', 'json'], { encoding: 'utf8', timeout: 20000, env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home, 'config'), XDG_CACHE_HOME: join(home, 'cache'), XDG_DATA_HOME: join(home, 'data'), GHIDRA_INSTALL_DIR: install }, }); if (run.error) throw run.error; const ghidra = JSON.parse(run.stdout).providerInspections.find(p => p.id === 'ghidra'); const version = ghidra.checks.find(c => c.name === 'version'); console.log(JSON.stringify({ name, exit: run.status, providerVersion: ghidra.providerVersion, version: { ok: version.ok, code: version.code, detail: version.detail } })); } } finally { await rm(root, { recursive: true, force: true }); } JS ``` Observed output, one JSON line per fixture: ```text {"name":"continued_hash","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}} {"name":"continued_bang","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}} {"name":"escaped_hash","exit":1,"providerVersion":null,"version":{"ok":false,"code":"version_unresolved","detail":"application.properties could not be parsed: Malformed Unicode escape in Java properties file"}} {"name":"real_comment","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}} {"name":"corrected","exit":1,"providerVersion":"12.1.4","version":{"ok":true,"code":null,"detail":"12.1.4"}} ``` ### Expected behavior Doctor should report `providerVersion: null` and a failed `version` check with code `version_unresolved` for both the continued-`#` and continued-`!` fixtures, exactly as it already does for the `escaped_hash` control, because the joined logical property contains a malformed Unicode escape that the documented parser contract must reject. ### Impact and scope Doctor falsely reports a passing version check and retains a provider version for a caller-managed `application.properties` that the documented parser contract requires it to reject; the failure is bounded to this continuation-into-comment-marker form, with no data loss or provider admission demonstrated. Workaround: available. A well-formed `application.properties` without a comment-marker continuation is validated correctly, as the `escaped_hash`, `corrected`, and `real_comment` controls show; correcting the malformed escape to `\u0041` preserves the property and restores correct validation. Reporter's impact estimate: P2 (supported). ### REA version 6.5.0, built from source at main commit 034342ba28c8e56c3746be92b78aa7299afa02bd (latest npm release is 6.5.0) ### Runtime and platform Node.js v26.10.0; macOS 27.0.1 arm64 ### Target and artifact context Ghidra application.properties fixture (metadata only) ### Interface CLI ### Earliest failure stage Setup or provider admission ### Relevant output or evidence - Expected source: src/ghidra/GhidraJavaProperties.ts:3 documents Java `Properties.load(InputStream)` semantics, under which a `#` or `!` that begins continuation data is property data rather than a physical comment; src/ghidra/GhidraInstallationPolicy.ts:54 requires a malformed escape to invalidate the whole file, and lines 64-70 clear the provider version when parsing fails. - Failure source: the observed output above, with src/ghidra/GhidraJavaProperties.ts:60 treating the marker-bearing physical line as continuation and line 73 joining it, after which the second comment check at line 11 drops the whole logical property before escape decoding, and src/ghidra/GhidraInstallation.ts:784 passes the retained release into the version check (loaded via src/cli/setupCommands.ts:124, src/doctorRuntime.ts:44, src/ghidra/GhidraDoctor.ts:62, and src/ghidra/GhidraInstallation.ts:423 and 481). - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete ### Typed outcome or recovery expected Doctor should have reported providerVersion: null and a failed version check with code version_unresolved, as it already does for the escaped-key control, instead of retaining 12.1.4 and reporting version.ok: true. Restoration check: Failing observable: the `continued_hash` and `continued_bang` fixtures yield `providerVersion: "12.1.4"` and `version.ok: true`. Passing observable: the same fixtures yield `providerVersion: null` and a failed `version` check with code `version_unresolved`, matching the `escaped_hash` control. Control: the `corrected` fixture (escape `\u0041`) and the `real_comment` fixture keep passing, confirming that well-formed inputs and genuine comments are unaffected. ### Analysis or suspected cause src/ghidra/GhidraJavaProperties.ts:60 and :73 join the comment-marker continuation line as continuation data, then the second comment check at :11 drops the whole logical property before escape decoding, so the malformed escape is never seen; src/ghidra/GhidraInstallation.ts:784 then passes the retained release into the version check. Reproduction path: src/cli/setupCommands.ts:124 -> src/doctorRuntime.ts:44 -> src/ghidra/GhidraDoctor.ts:62 -> src/ghidra/GhidraInstallation.ts:423 and :481. ### Additional context The audit tracker #3537 indexes closed #3179 but does not separately describe this defect, and closed #2785 concerns JVM architecture admission and is distinct. This report came from AI-assisted source review with local reproduction where stated above; the evidence labels distinguish observed runs from source-only reasoning.

bsakaguchi commented

2026-10-16 21:06:08 +00:00

Author

Owner

@yankikucuk commented on GitHub (Oct 15, 2026):

This repeats #3179 byte for byte: the same five fixtures (continued_hash, continued_bang, escaped_hash, real_comment, corrected), the same observed lines and the same expected outcome. #3179 was closed by the maintainer as expected Java Properties behaviour rather than a REA validation defect, based on the Java 25 Properties.load comparison in https://github.com/morluto/rea/issues/3179#issuecomment-6169112199: a physical line holding only a continuation backslash leaves no buffered content, so the following # or ! line starts a new comment line, and parseGhidraJavaProperties matches that. Unless this report has an input that differs from #3179, it can be closed as a duplicate.

@yankikucuk commented on GitHub (Oct 15, 2026): This repeats #3179 byte for byte: the same five fixtures (`continued_hash`, `continued_bang`, `escaped_hash`, `real_comment`, `corrected`), the same observed lines and the same expected outcome. #3179 was closed by the maintainer as expected Java Properties behaviour rather than a REA validation defect, based on the Java 25 `Properties.load` comparison in https://github.com/morluto/rea/issues/3179#issuecomment-6169112199: a physical line holding only a continuation backslash leaves no buffered content, so the following `#` or `!` line starts a new comment line, and `parseGhidraJavaProperties` matches that. Unless this report has an input that differs from #3179, it can be closed as a duplicate.

bsakaguchi referenced this issue

2026-10-17 21:11:54 +00:00

[PR #1506] [MERGED] fix(mobile): bound framework conventions to path segments #2926

bsakaguchi referenced this issue

2026-10-17 21:11:59 +00:00

[PR #1513] [MERGED] fix: bind identity and coverage to the producing rule #2933

bsakaguchi closed this issue

2026-10-31 00:31:01 +00:00

bsakaguchi commented

2026-10-31 00:31:01 +00:00

Author

Owner

@gensokyo-dev commented on GitHub (Oct 20, 2026):

Qualified on current main 052956af3416fa01d93837e54fdec3ad2739f6d1 against native JDK 21 Properties.load(InputStream). The reported Java/REA mismatch does not reproduce: Java itself treats the literal #/! after this empty continuation as a comment.

Minimal native Java reproduction (e.g. in JShell):

var p = new java.util.Properties();
p.load(new java.io.ByteArrayInputStream("application.version=12.1.4\n\\\n#unused=\\uZZZZ\n".getBytes(java.nio.charset.StandardCharsets.ISO_8859_1)));
System.out.println(p); // {application.version=12.1.4}

Executed controls:

Input	Native Java 21	Public REA doctor version check
Empty continuation → literal `#` or `!`, malformed escape	Loads; comment omitted	Passes, version 12.1.4
Escaped `\#` key, malformed escape	Throws `IllegalArgumentException`	`version_unresolved`, version null
Nonempty `key=` continuation → `#`, malformed escape	Throws `IllegalArgumentException`	`version_unresolved`, version null
Natural comments / corrected fixture	Loads	Passes, version 12.1.4
Official Ghidra 12.1.4 metadata	Loads	Healthy provider, doctor exit 0

REA matched all 18 complete property-map/error results, including LF/CR/CRLF, leading continuation whitespace, multiple/ended continuations, escaped markers and the real installation metadata. Seven public CLI calls verified the version projection and real-provider readiness. Metadata-only fixtures still exit 1 because they omit other installation prerequisites, as in the report.

The original real installation was unchanged; its application.properties SHA-256 is e84c26b08545eee63d24a1a8a4dd7905e4209538e2e7192093877c435c64eb67. Completed qualification: one CPU, 22.249 s elapsed, 20.358 s process-family CPU, 447.6 MiB peak, no OOM and joined cleanup. This verifies metadata/readiness with Ghidra 12.1.4 and JDK 21, not target analysis or every JDK major.

Closing as not planned because the expected difference contradicts the tested native producer. No parser change is needed for these inputs. An initial proposed regression expectation was discarded after the independent native oracle contradicted it; no implementation/test change was integrated.

@gensokyo-dev commented on GitHub (Oct 20, 2026): Qualified on current main `052956af3416fa01d93837e54fdec3ad2739f6d1` against native JDK 21 `Properties.load(InputStream)`. The reported Java/REA mismatch does not reproduce: Java itself treats the literal `#`/`!` after this **empty** continuation as a comment. Minimal native Java reproduction (e.g. in JShell): ```java var p = new java.util.Properties(); p.load(new java.io.ByteArrayInputStream("application.version=12.1.4\n\\\n#unused=\\uZZZZ\n".getBytes(java.nio.charset.StandardCharsets.ISO_8859_1))); System.out.println(p); // {application.version=12.1.4} ``` Executed controls: | Input | Native Java 21 | Public REA doctor version check | | --- | --- | --- | | Empty continuation → literal `#` or `!`, malformed escape | Loads; comment omitted | Passes, version 12.1.4 | | Escaped `\#` key, malformed escape | Throws `IllegalArgumentException` | `version_unresolved`, version null | | Nonempty `key=` continuation → `#`, malformed escape | Throws `IllegalArgumentException` | `version_unresolved`, version null | | Natural comments / corrected fixture | Loads | Passes, version 12.1.4 | | Official Ghidra 12.1.4 metadata | Loads | Healthy provider, doctor exit 0 | REA matched **all 18 complete property-map/error results**, including LF/CR/CRLF, leading continuation whitespace, multiple/ended continuations, escaped markers and the real installation metadata. Seven public CLI calls verified the version projection and real-provider readiness. Metadata-only fixtures still exit 1 because they omit other installation prerequisites, as in the report. The original real installation was unchanged; its `application.properties` SHA-256 is `e84c26b08545eee63d24a1a8a4dd7905e4209538e2e7192093877c435c64eb67`. Completed qualification: one CPU, 22.249 s elapsed, 20.358 s process-family CPU, 447.6 MiB peak, no OOM and joined cleanup. This verifies metadata/readiness with Ghidra 12.1.4 and JDK 21, not target analysis or every JDK major. Closing as not planned because the expected difference contradicts the tested native producer. No parser change is needed for these inputs. An initial proposed regression expectation was discarded after the independent native oracle contradicted it; no implementation/test change was integrated.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1489

No description provided.

Rows
Columns