morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 07:47:29 +00:00

[GH-ISSUE #3361] [Bug] Recovered invalid templates become resolved semantic strings #1230

New issue

Closed

opened 2026-10-16 21:03:04 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:03:04 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3361

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

After recovering invalid Unicode escapes in template literals, analyze-javascript-application reports raw template spellings as concrete child-process commands, request endpoints, and event names in normalized_result.semantic_graph. The values have no valid cooked JavaScript string, but their spawns, constructs-request, and registers-listener relations are resolved.

The observed CLI run exits with code 0 and retains \unicode as the child command and event name, and https://example.invalid/\unicode as the request endpoint. Overall partial/unknown graph coverage does not mark these concrete values as unavailable. Parsing the same source with the installed Babel parser and error recovery reports three invalid-template-escape errors and cooked: null for all three literals. A separate valid-template control recovers program, https://example.invalid/valid, and ready with the corresponding resolved relations. Local verification reproduced the failure with a complete reproduction.

Reproduction

Start from a clean checkout of https://github.com/morluto/rea and run these commands from the repository root.

git checkout --detach 8dde6370d0809ad04cda3342d4e6f290f48aa063
npm ci
npm run build:cached

Create a disposable input and isolated configuration home, run the public CLI, and print the affected facts from its complete Evidence. The analyzer reads the target as text and does not execute its calls. The final command parses the same input through the installed Babel parser as a cooked-value control.

work=$(mktemp -d)
mkdir "$work/app" "$work/home"
cat > "$work/app/main.mjs" <<'JS'
import { spawn } from 'node:child_process';
spawn(`\unicode`, []);
fetch(`https://example.invalid/\unicode`);
bus.on(`\unicode`, listener);
JS
python3 - "$work" <<'PY'
import json, os, pathlib, subprocess, sys
work = pathlib.Path(sys.argv[1])
child_environment = dict(os.environ, HOME=str(work / 'home'))
result = subprocess.run(
    ['node', 'scripts/rea.mjs', 'analyze-javascript-application',
     str(work / 'app'), '--format', 'json'],
    env=child_environment, capture_output=True, text=True, check=True)
evidence = json.loads(result.stdout)
(work / 'evidence.json').write_text(result.stdout)
graph = evidence['normalized_result']['semantic_graph']
print('exit:', result.returncode)
nodes = {node['node_id']: node for node in graph['nodes']}
for node in sorted(graph['nodes'], key=lambda n: ((n['identity']['source_range'] or {}).get('start', {}).get('line', 0), n['node_id'])):
    if node['kind'] in ['child-process', 'request', 'event']:
        print('node', node['kind'], node['identity']['source_range'], node['properties'])
for relation in graph['relations']:
    if relation['relation'] in ['spawns', 'constructs-request', 'registers-listener']:
        source = nodes[relation['source_node_id']]
        target = nodes[relation['target_node_id']]
        print('relation', relation['relation'], relation['resolution'], source['kind'], source['node_id'], source['identity']['source_range'], '=>', target['kind'], target['node_id'], target['identity']['source_range'])
print('effect_unknowns', [(u['family'], u['detail']) for u in graph['unknowns'] if u['family'] in ['timer', 'child-process', 'event', 'request', 'boundary']])
PY
node --input-type=module - "$work/app/main.mjs" <<'JS'
import { readFileSync } from 'node:fs';
import { parse } from '@babel/parser';
const file = parse(readFileSync(process.argv[2], 'utf8'), {
  sourceType: 'module', errorRecovery: true
});
console.log(file.errors.map(error => error.message));
console.log(file.program.body.slice(1).map(statement =>
  statement.expression.arguments[0].quasis[0].value));
JS

For the valid-template control, replace the input with the following content and rerun the Python analysis and Babel parsing blocks from step 2 against the same input directory and file.

cat > "$work/app/main.mjs" <<'JS'
import { spawn } from 'node:child_process';
spawn(`program`, []);
fetch(`https://example.invalid/valid`);
bus.on(`ready`, listener);
JS

Compare the invalid-template values and relation resolutions with the valid control. The observed invalid case has three invalid-template-escape errors and null cooked values, yet produces concrete semantic strings and resolved relations. The observed valid control recovers the exact command, endpoint, and event strings with the corresponding resolved relations.

Expected behavior

Recovered templates with a parser error and cooked: null must remain dynamic or unknown. They cannot establish a JavaScript command string, endpoint, or event name. Preserve source provenance and explicit parser/value uncertainty rather than replacing an unavailable value with its raw spelling.

docs/javascript-artifact-reconstruction.md:238-242 explicitly states that a template recovered with a parser error and no cooked value stays dynamic and that its raw spelling is not a valid JavaScript string. docs/javascript-application-workflows.md:96-106 preserves parser-recovery unknowns.

Impact and scope

Impact level: P2. Assessment status: supported. Malformed but recoverable input supplies fabricated semantic strings and exact relationships, affecting literal searches and child-process, request, and event analysis.

Workaround status: unverified. No analysis option preserving the original malformed source and preventing only the fabricated semantic values was verified. Editing the escape changes the source and cannot establish what its unavailable cooked value was.

REA version

6.4.0 (main at 8dde6370d0)

Runtime and platform

Node.js v26.10.0; macOS 27.0.1. Architecture was not recorded in the reproduction evidence. Dependencies were installed and the runtime was built.

Target and artifact context

A local directory containing the inline JavaScript fixture, analyzed as text. The affected Evidence is normalized_result.semantic_graph, including command/endpoint/event properties and their corresponding resolved relations. No native engine, browser, or external service is needed.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

The following summarizes the observed values and resolutions; it is not a verbatim CLI transcript.

CLI exit code: 0
Child command: \unicode
Request endpoint: https://example.invalid/\unicode
Event name: \unicode
spawns: resolved
constructs-request: resolved
registers-listener: resolved
Babel error recovery: three invalid-template-escape errors
Babel cooked values: null for all three templates
Valid-template control: program, https://example.invalid/valid, ready
Valid-template control relations: resolved

Typed outcome or recovery expected

Rerun the fixture. Invalid template values must not become concrete semantic command, endpoint, or event strings, and dependent relationships must retain explicit uncertainty. Preserve the recovered source provenance and distinguish unavailable values from exact strings.

As a discriminating control, replace each invalid escape with the valid literal strings in step 3 and verify that program, https://example.invalid/valid, and ready and their eligible resolved relationships remain recoverable.

Analysis or suspected cause

Source-derived reasoning: src/domain/javascript/javascriptSemanticDataEffectHelpers.ts:928-941 falls back to raw template spelling when exact literal recovery fails in dataEffectLiteralString. That fallback treats a recovered template with no cooked value as a concrete string. The executed analyze-javascript-application Evidence contains the resulting command/endpoint/event properties and corresponding resolved relations.

Additional context

Related: #2627, closed as completed, concerns lost parser-recovery completeness metadata in captured-bundle results; this failure fabricates semantic literal values even when overall graph coverage is partial or unknown.

Related: #2572, closed as completed, concerns an unknown child kill signal defaulting to SIGTERM; this failure comes from raw template fallback for command, endpoint, and event strings.

Related: #2805, #2841, and #2599, all closed as completed, concern omitted valid expression-free template routes or imports; this report concerns malformed templates with no valid cooked value becoming concrete semantic strings.

Originally created by @coygeek on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3361 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior After recovering invalid Unicode escapes in template literals, `analyze-javascript-application` reports raw template spellings as concrete child-process commands, request endpoints, and event names in `normalized_result.semantic_graph`. The values have no valid cooked JavaScript string, but their `spawns`, `constructs-request`, and `registers-listener` relations are resolved. The observed CLI run exits with code 0 and retains `\unicode` as the child command and event name, and `https://example.invalid/\unicode` as the request endpoint. Overall partial/unknown graph coverage does not mark these concrete values as unavailable. Parsing the same source with the installed Babel parser and error recovery reports three invalid-template-escape errors and `cooked: null` for all three literals. A separate valid-template control recovers `program`, `https://example.invalid/valid`, and `ready` with the corresponding resolved relations. Local verification reproduced the failure with a complete reproduction. ### Reproduction 1. Start from a clean checkout of `https://github.com/morluto/rea` and run these commands from the repository root. ```sh git checkout --detach 8dde6370d0809ad04cda3342d4e6f290f48aa063 npm ci npm run build:cached ``` 2. Create a disposable input and isolated configuration home, run the public CLI, and print the affected facts from its complete Evidence. The analyzer reads the target as text and does not execute its calls. The final command parses the same input through the installed Babel parser as a cooked-value control. ```bash work=$(mktemp -d) mkdir "$work/app" "$work/home" cat > "$work/app/main.mjs" <<'JS' import { spawn } from 'node:child_process'; spawn(`\unicode`, []); fetch(`https://example.invalid/\unicode`); bus.on(`\unicode`, listener); JS python3 - "$work" <<'PY' import json, os, pathlib, subprocess, sys work = pathlib.Path(sys.argv[1]) child_environment = dict(os.environ, HOME=str(work / 'home')) result = subprocess.run( ['node', 'scripts/rea.mjs', 'analyze-javascript-application', str(work / 'app'), '--format', 'json'], env=child_environment, capture_output=True, text=True, check=True) evidence = json.loads(result.stdout) (work / 'evidence.json').write_text(result.stdout) graph = evidence['normalized_result']['semantic_graph'] print('exit:', result.returncode) nodes = {node['node_id']: node for node in graph['nodes']} for node in sorted(graph['nodes'], key=lambda n: ((n['identity']['source_range'] or {}).get('start', {}).get('line', 0), n['node_id'])): if node['kind'] in ['child-process', 'request', 'event']: print('node', node['kind'], node['identity']['source_range'], node['properties']) for relation in graph['relations']: if relation['relation'] in ['spawns', 'constructs-request', 'registers-listener']: source = nodes[relation['source_node_id']] target = nodes[relation['target_node_id']] print('relation', relation['relation'], relation['resolution'], source['kind'], source['node_id'], source['identity']['source_range'], '=>', target['kind'], target['node_id'], target['identity']['source_range']) print('effect_unknowns', [(u['family'], u['detail']) for u in graph['unknowns'] if u['family'] in ['timer', 'child-process', 'event', 'request', 'boundary']]) PY node --input-type=module - "$work/app/main.mjs" <<'JS' import { readFileSync } from 'node:fs'; import { parse } from '@babel/parser'; const file = parse(readFileSync(process.argv[2], 'utf8'), { sourceType: 'module', errorRecovery: true }); console.log(file.errors.map(error => error.message)); console.log(file.program.body.slice(1).map(statement => statement.expression.arguments[0].quasis[0].value)); JS ``` 3. For the valid-template control, replace the input with the following content and rerun the Python analysis and Babel parsing blocks from step 2 against the same input directory and file. ```sh cat > "$work/app/main.mjs" <<'JS' import { spawn } from 'node:child_process'; spawn(`program`, []); fetch(`https://example.invalid/valid`); bus.on(`ready`, listener); JS ``` 4. Compare the invalid-template values and relation resolutions with the valid control. The observed invalid case has three invalid-template-escape errors and null cooked values, yet produces concrete semantic strings and resolved relations. The observed valid control recovers the exact command, endpoint, and event strings with the corresponding resolved relations. ### Expected behavior Recovered templates with a parser error and `cooked: null` must remain dynamic or unknown. They cannot establish a JavaScript command string, endpoint, or event name. Preserve source provenance and explicit parser/value uncertainty rather than replacing an unavailable value with its raw spelling. `docs/javascript-artifact-reconstruction.md:238-242` explicitly states that a template recovered with a parser error and no cooked value stays dynamic and that its raw spelling is not a valid JavaScript string. `docs/javascript-application-workflows.md:96-106` preserves parser-recovery unknowns. ### Impact and scope Impact level: P2. Assessment status: supported. Malformed but recoverable input supplies fabricated semantic strings and exact relationships, affecting literal searches and child-process, request, and event analysis. Workaround status: unverified. No analysis option preserving the original malformed source and preventing only the fabricated semantic values was verified. Editing the escape changes the source and cannot establish what its unavailable cooked value was. ### REA version 6.4.0 (main at 8dde6370d0809ad04cda3342d4e6f290f48aa063) ### Runtime and platform Node.js v26.10.0; macOS 27.0.1. Architecture was not recorded in the reproduction evidence. Dependencies were installed and the runtime was built. ### Target and artifact context A local directory containing the inline JavaScript fixture, analyzed as text. The affected Evidence is `normalized_result.semantic_graph`, including command/endpoint/event properties and their corresponding resolved relations. No native engine, browser, or external service is needed. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence The following summarizes the observed values and resolutions; it is not a verbatim CLI transcript. ```text CLI exit code: 0 Child command: \unicode Request endpoint: https://example.invalid/\unicode Event name: \unicode spawns: resolved constructs-request: resolved registers-listener: resolved Babel error recovery: three invalid-template-escape errors Babel cooked values: null for all three templates Valid-template control: program, https://example.invalid/valid, ready Valid-template control relations: resolved ``` ### Typed outcome or recovery expected Rerun the fixture. Invalid template values must not become concrete semantic command, endpoint, or event strings, and dependent relationships must retain explicit uncertainty. Preserve the recovered source provenance and distinguish unavailable values from exact strings. As a discriminating control, replace each invalid escape with the valid literal strings in step 3 and verify that `program`, `https://example.invalid/valid`, and `ready` and their eligible resolved relationships remain recoverable. ### Analysis or suspected cause Source-derived reasoning: `src/domain/javascript/javascriptSemanticDataEffectHelpers.ts:928-941` falls back to raw template spelling when exact literal recovery fails in `dataEffectLiteralString`. That fallback treats a recovered template with no cooked value as a concrete string. The executed `analyze-javascript-application` Evidence contains the resulting command/endpoint/event properties and corresponding resolved relations. ### Additional context Related: [#2627](https://github.com/morluto/rea/issues/2627), closed as completed, concerns lost parser-recovery completeness metadata in captured-bundle results; this failure fabricates semantic literal values even when overall graph coverage is partial or unknown. Related: [#2572](https://github.com/morluto/rea/issues/2572), closed as completed, concerns an unknown child kill signal defaulting to SIGTERM; this failure comes from raw template fallback for command, endpoint, and event strings. Related: [#2805](https://github.com/morluto/rea/issues/2805), [#2841](https://github.com/morluto/rea/issues/2841), and [#2599](https://github.com/morluto/rea/issues/2599), all closed as completed, concern omitted valid expression-free template routes or imports; this report concerns malformed templates with no valid cooked value becoming concrete semantic strings.

bsakaguchi closed this issue

2026-10-16 21:03:04 +00:00

bsakaguchi referenced this issue

2026-10-17 21:09:43 +00:00

[PR #1230] [MERGED] fix(mcp): emit portable Unicode NUL patterns #2740