morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #3534] [CLOSED] fix(javascript): keep unresolved network request targets visible as partial endpoints #5810

New issue

Closed

opened 2026-10-31 01:09:35 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 01:09:35 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/3534
Author: @hemv-857
Created: 10/14/2026
Status: ❌ Closed

Base: main ← Head: fix-3451-partial-network-endpoints

📝 Commits (3)

1d78bdf fix(javascript): keep unresolved network request targets visible as partial endpoints
8b97278 fix(javascript): harden partial endpoint resolution from review
06dd652 Merge remote-tracking branch 'origin/main' into sweep/3534

📊 Changes

10 files changed (+477 additions, -16 deletions)

View changed files

📝 src/application/javascript/JavaScriptArtifactGraphFindings.ts (+16 -2)
📝 src/domain/javascript/javascriptStaticAnalysis.event_source.test.ts (+15 -3)
➕ src/domain/javascript/javascriptStaticAnalysis.partialEndpoints.test.ts (+161 -0)
📝 src/domain/javascript/javascriptStaticAnalysis.send_beacon.test.ts (+15 -3)
📝 src/domain/javascript/javascriptStaticAnalysisCalls.ts (+166 -8)
📝 src/domain/javascript/javascriptStaticAnalysisState.ts (+1 -0)
📝 src/domain/javascript/javascriptStaticAnalysisTypes.ts (+2 -0)
📝 src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.test.ts (+16 -0)
📝 src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts (+1 -0)
📝 tests/boundary/filesystem/javascript-applications/javascriptArtifactGraphEndpoints.test.ts (+84 -0)

📄 Description

Fixes #3451

Summary

analyze-javascript-application reports a network endpoint observation only when a recognized request call's target is an exact string literal. Any other construction (template literal, variable base, URL object, config property) produced no endpoint observation at all, so an application whose requests are all dynamically shaped read as having no first-party backend. Non-literal request targets of unambiguous network APIs are now retained as partial endpoints carrying the unresolved source expression, and targets that resolve through the existing constant lattice (config properties, new URL(...) bindings) resolve to complete endpoints.

Problem and expected behavior

Smallest trigger (from the issue):

fetch(`https://core.example.com/notes/${id}/audio/`);

Observed before: no endpoint observation; the application graph contained no network endpoint node for the call site. Violated invariant: a recognized request call site must remain visible even when its target is not statically exact.

Expected: the call site produces a network endpoint observation with the unresolved part explicit. Constant shapes resolve fully:

const cfg = { wsUrl: "wss://config.example.com" };
new WebSocket(cfg.wsUrl); // → wss://config.example.com
const u = new URL("wss://realtime.example.com");
new WebSocket(u);
new WebSocket(u.toString()); // → wss://realtime.example.com

Change and scope

One fallback at the single static-analysis choke point inspectEndpointCall (src/domain/javascript/javascriptStaticAnalysisCalls.ts), reusing existing machinery:

evaluateSemanticExpression — the bounded constant lattice the semantic analysis already uses. Targets that evaluate to string literals resolve to complete endpoints.
A narrow URL-object unwrap: a single immutable binding initialized by new URL(...), forwarded directly or through zero-argument .toString(), following the new URL unwrapping precedent in workerUrlSpecifier in the same file. A single literal argument matches the literal path's exact text; a two-argument new URL(input, base) with literal arguments resolves through the platform URL constructor; a non-literal base, a member write anywhere in the program (u.hostname = ...; the static pass does not collect member mutations, so writes are detected syntactically once per parse), or any other irregularity keeps the target partial rather than reporting a possibly wrong complete URL.
sourceSlice — the same specifier/expression retention pattern as JavaScriptStaticReference and ElectronStaticValue. URL userinfo is stripped from retained expression text before storage; interpolations stay untouched.

Applied to unambiguous network APIs: bare fetch, WebSocket, EventSource, member EventSource, sendBeacon, member sendBeacon. Member .fetch is excluded from the partial fallback — local cache and service-worker style methods share the name without making a request, and the literal path's pre-existing behavior is unchanged. HTTP-method-named calls (get, post, request, ...) are excluded on purpose: their first argument is frequently a collection or setting key (map.get(key), configuration reads), and emitting partial endpoints there would flood results with false endpoints.

Partial endpoints carry a disjoint "partial" segment in their artifact-local identity, so a partial expression and a complete literal with the same text (fetch(target) vs fetch("target")) never collapse into one node; complete endpoint identities are unchanged.

Intentionally not included:

The semantic graph's requestEndpoint reader (javascriptSemanticDataEffects.ts) remains literal-only. That surface already reports these call sites as request operations with resolution: "partial" and an explicit unknown endpoint; carrying the partial text there touches the semantic IR with a wider blast radius and belongs in a follow-up.
Route-property templates are a different endpoint kind with their own guard and are unchanged.
analyze_web_bundle has a separate endpoint-candidate path and is unchanged.

Contribution scope: ordinary bug fix within existing static-analysis boundaries. Maintainer scope decision: not applicable.

Contract and boundary impact

Semantic owner and earliest changed stage: static JavaScript analysis domain (javascriptStaticAnalysisCalls.ts), consumed by the application-graph projection and the worker schema.
Contribution scope fit and linked maintainer decision: fix within existing boundaries; not applicable.
CLI and MCP/tool-catalog contract: no tool input/output schema changes. Advertised JSON Schemas and catalog digests do not move; endpoint observations are result data.
Provider, bridge, target-format, or platform compatibility: none; pure static AST analysis.
Evidence, artifact, provenance, or reconstruction contract: partial endpoints keep ast-static-analysis / observed / exact evidence for the call site itself, with an added limitation that the value is an unresolved source expression. Node identity is unchanged: partial values are source expressions and complete values are resolved URLs, so they cannot collide.
Target selection, authority, upstream behavior, and cleanup details: not applicable.
Generated metadata impact: none; regenerated catalogs are ignored outputs and do not change from this diff.

Expected effects:

Read existing files or captured data.
Launch or attach to a process.
Inject instrumentation or run scripts/code.
Change process memory, registers, files, or configuration.
Contact network services.
Recover protected code or work around protections.
Documentation or usability only; no target interaction.
Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Tests added or updated:
- New src/domain/javascript/javascriptStaticAnalysis.partialEndpoints.test.ts (11 tests) — all four shapes from the issue (interpolated template, variable base, URL object direct and through .toString(), config property), two-argument URL bases with literal and non-literal bases, mutated URL objects, credential stripping in retained expressions, member .fetch exclusion, and a negative for collection reads.
- Updated javascriptStaticAnalysis.event_source.test.ts and javascriptStaticAnalysis.send_beacon.test.ts — the two tests that pinned "dynamic URL unreported" now assert the partial observation; they pinned the reported bug.
- Boundary tests in javascriptArtifactGraphEndpoints.test.ts — partial node/edge properties with evidence and limitation through buildJavaScriptArtifactGraph, and distinct node identities for fetch(target) vs fetch("target").
- New worker-schema round-trip in JavaScriptAnalysisWorkerSchemas.test.ts — resolution: "partial" survives the strictObject worker handoff.
Base reproduction or other evidence: on the affected base (origin/main at 3b79af2), the partial cases produce no endpoint observation at all (the dynamic EventSource and beacon tests asserted []), which is exactly the reported omission; the complete-resolution shapes (config property, URL object) also produced nothing on base.
User-visible CLI/MCP output: analyze_javascript_application graph output gains endpoint nodes with properties.resolution: "partial" and the limitation text; no CLI flag or schema change.
Remaining proof gaps: this is static AST analysis only; no browser or client-chat runtime reproduction is claimed.

For evidence-bearing changes:

Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

npx vitest run src/domain/javascript/ src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.test.ts tests/boundary/filesystem/javascript-applications/javascriptArtifactGraphEndpoints.test.ts tests/boundary/filesystem/javascript-applications/javascriptApplicationEmptyLiterals.test.ts tests/acceptance/applications/javascriptEndpointCandidateCli.test.ts tests/acceptance/applications/javascriptEndpointPayload.test.ts — 74 files, 1674 tests passed.
npm run check:fast — typecheck + lint successful.
npx tsc --noEmit — clean.
npx oxfmt --check on changed files — clean.
npm run verify:test-discovery — 1276 test files discovered exactly once.

Compatibility, safety, and release

Breaking changes or migration steps: none. Existing complete endpoint observations are unchanged; new partial observations are additive; node identity keys are unchanged.
Real Hopper/Ghidra, browser, or OS coverage: not applicable (static JavaScript analysis).
Package or release metadata impact: none.
Security, privacy, process, or containment review: partial endpoint values are source expressions already present in the analyzed artifact; no new data leaves the analysis boundary. sanitizeEndpointCandidate continues to pass unparseable expressions through verbatim.

Author readiness

Human owner: @hemv-857

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/3534 **Author:** [@hemv-857](https://github.com/hemv-857) **Created:** 10/14/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix-3451-partial-network-endpoints` --- ### 📝 Commits (3) - [`1d78bdf`](https://github.com/morluto/rea/commit/1d78bdfff1a0a5cd495b235046c68c9e277fc404) fix(javascript): keep unresolved network request targets visible as partial endpoints - [`8b97278`](https://github.com/morluto/rea/commit/8b97278516a72c7b52a5ef894a1417a77b26a6c9) fix(javascript): harden partial endpoint resolution from review - [`06dd652`](https://github.com/morluto/rea/commit/06dd652c9c861de1bfd80f63d9e8f032958ac78f) Merge remote-tracking branch 'origin/main' into sweep/3534 ### 📊 Changes **10 files changed** (+477 additions, -16 deletions) <details> <summary>View changed files</summary> 📝 `src/application/javascript/JavaScriptArtifactGraphFindings.ts` (+16 -2) 📝 `src/domain/javascript/javascriptStaticAnalysis.event_source.test.ts` (+15 -3) ➕ `src/domain/javascript/javascriptStaticAnalysis.partialEndpoints.test.ts` (+161 -0) 📝 `src/domain/javascript/javascriptStaticAnalysis.send_beacon.test.ts` (+15 -3) 📝 `src/domain/javascript/javascriptStaticAnalysisCalls.ts` (+166 -8) 📝 `src/domain/javascript/javascriptStaticAnalysisState.ts` (+1 -0) 📝 `src/domain/javascript/javascriptStaticAnalysisTypes.ts` (+2 -0) 📝 `src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.test.ts` (+16 -0) 📝 `src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.ts` (+1 -0) 📝 `tests/boundary/filesystem/javascript-applications/javascriptArtifactGraphEndpoints.test.ts` (+84 -0) </details> ### 📄 Description Fixes #3451 ## Summary `analyze-javascript-application` reports a `network` endpoint observation only when a recognized request call's target is an exact string literal. Any other construction (template literal, variable base, URL object, config property) produced no endpoint observation at all, so an application whose requests are all dynamically shaped read as having no first-party backend. Non-literal request targets of unambiguous network APIs are now retained as partial endpoints carrying the unresolved source expression, and targets that resolve through the existing constant lattice (config properties, `new URL(...)` bindings) resolve to complete endpoints. ## Problem and expected behavior Smallest trigger (from the issue): ```js fetch(`https://core.example.com/notes/${id}/audio/`); ``` Observed before: no endpoint observation; the application graph contained no `network` endpoint node for the call site. Violated invariant: a recognized request call site must remain visible even when its target is not statically exact. Expected: the call site produces a `network` endpoint observation with the unresolved part explicit. Constant shapes resolve fully: ```js const cfg = { wsUrl: "wss://config.example.com" }; new WebSocket(cfg.wsUrl); // → wss://config.example.com const u = new URL("wss://realtime.example.com"); new WebSocket(u); new WebSocket(u.toString()); // → wss://realtime.example.com ``` ## Change and scope One fallback at the single static-analysis choke point `inspectEndpointCall` (`src/domain/javascript/javascriptStaticAnalysisCalls.ts`), reusing existing machinery: - `evaluateSemanticExpression` — the bounded constant lattice the semantic analysis already uses. Targets that evaluate to string literals resolve to complete endpoints. - A narrow URL-object unwrap: a single immutable binding initialized by `new URL(...)`, forwarded directly or through zero-argument `.toString()`, following the `new URL` unwrapping precedent in `workerUrlSpecifier` in the same file. A single literal argument matches the literal path's exact text; a two-argument `new URL(input, base)` with literal arguments resolves through the platform URL constructor; a non-literal base, a member write anywhere in the program (`u.hostname = ...`; the static pass does not collect member mutations, so writes are detected syntactically once per parse), or any other irregularity keeps the target partial rather than reporting a possibly wrong complete URL. - `sourceSlice` — the same specifier/expression retention pattern as `JavaScriptStaticReference` and `ElectronStaticValue`. URL userinfo is stripped from retained expression text before storage; interpolations stay untouched. Applied to unambiguous network APIs: bare `fetch`, `WebSocket`, `EventSource`, member `EventSource`, `sendBeacon`, member `sendBeacon`. Member `.fetch` is excluded from the partial fallback — local cache and service-worker style methods share the name without making a request, and the literal path's pre-existing behavior is unchanged. HTTP-method-named calls (`get`, `post`, `request`, ...) are excluded on purpose: their first argument is frequently a collection or setting key (`map.get(key)`, configuration reads), and emitting partial endpoints there would flood results with false endpoints. Partial endpoints carry a disjoint `"partial"` segment in their artifact-local identity, so a partial expression and a complete literal with the same text (`fetch(target)` vs `fetch("target")`) never collapse into one node; complete endpoint identities are unchanged. Intentionally not included: - The semantic graph's `requestEndpoint` reader (`javascriptSemanticDataEffects.ts`) remains literal-only. That surface already reports these call sites as request operations with `resolution: "partial"` and an explicit unknown endpoint; carrying the partial text there touches the semantic IR with a wider blast radius and belongs in a follow-up. - Route-property templates are a different endpoint kind with their own guard and are unchanged. - `analyze_web_bundle` has a separate endpoint-candidate path and is unchanged. Contribution scope: ordinary bug fix within existing static-analysis boundaries. Maintainer scope decision: not applicable. ## Contract and boundary impact - Semantic owner and earliest changed stage: static JavaScript analysis domain (`javascriptStaticAnalysisCalls.ts`), consumed by the application-graph projection and the worker schema. - Contribution scope fit and linked maintainer decision: fix within existing boundaries; not applicable. - CLI and MCP/tool-catalog contract: no tool input/output schema changes. Advertised JSON Schemas and catalog digests do not move; endpoint observations are result data. - Provider, bridge, target-format, or platform compatibility: none; pure static AST analysis. - Evidence, artifact, provenance, or reconstruction contract: partial endpoints keep `ast-static-analysis` / `observed` / `exact` evidence for the call site itself, with an added limitation that the value is an unresolved source expression. Node identity is unchanged: partial values are source expressions and complete values are resolved URLs, so they cannot collide. - Target selection, authority, upstream behavior, and cleanup details: not applicable. - Generated metadata impact: none; regenerated catalogs are ignored outputs and do not change from this diff. Expected effects: - [x] Read existing files or captured data. - [x] Launch or attach to a process. - [x] Inject instrumentation or run scripts/code. - [x] Change process memory, registers, files, or configuration. - [x] Contact network services. - [x] Recover protected code or work around protections. - [x] Documentation or usability only; no target interaction. - [x] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: - New `src/domain/javascript/javascriptStaticAnalysis.partialEndpoints.test.ts` (11 tests) — all four shapes from the issue (interpolated template, variable base, URL object direct and through `.toString()`, config property), two-argument URL bases with literal and non-literal bases, mutated URL objects, credential stripping in retained expressions, member `.fetch` exclusion, and a negative for collection reads. - Updated `javascriptStaticAnalysis.event_source.test.ts` and `javascriptStaticAnalysis.send_beacon.test.ts` — the two tests that pinned "dynamic URL unreported" now assert the partial observation; they pinned the reported bug. - Boundary tests in `javascriptArtifactGraphEndpoints.test.ts` — partial node/edge properties with evidence and limitation through `buildJavaScriptArtifactGraph`, and distinct node identities for `fetch(target)` vs `fetch("target")`. - New worker-schema round-trip in `JavaScriptAnalysisWorkerSchemas.test.ts` — `resolution: "partial"` survives the strictObject worker handoff. - Base reproduction or other evidence: on the affected base (`origin/main` at `3b79af2`), the partial cases produce no endpoint observation at all (the dynamic EventSource and beacon tests asserted `[]`), which is exactly the reported omission; the complete-resolution shapes (config property, URL object) also produced nothing on base. - User-visible CLI/MCP output: `analyze_javascript_application` graph output gains endpoint nodes with `properties.resolution: "partial"` and the limitation text; no CLI flag or schema change. - Remaining proof gaps: this is static AST analysis only; no browser or client-chat runtime reproduction is claimed. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed - `npx vitest run src/domain/javascript/ src/javascript/analysis/JavaScriptAnalysisWorkerSchemas.test.ts tests/boundary/filesystem/javascript-applications/javascriptArtifactGraphEndpoints.test.ts tests/boundary/filesystem/javascript-applications/javascriptApplicationEmptyLiterals.test.ts tests/acceptance/applications/javascriptEndpointCandidateCli.test.ts tests/acceptance/applications/javascriptEndpointPayload.test.ts` — 74 files, 1674 tests passed. - `npm run check:fast` — typecheck + lint successful. - `npx tsc --noEmit` — clean. - `npx oxfmt --check` on changed files — clean. - `npm run verify:test-discovery` — 1276 test files discovered exactly once. ## Compatibility, safety, and release - Breaking changes or migration steps: none. Existing complete endpoint observations are unchanged; new partial observations are additive; node identity keys are unchanged. - Real Hopper/Ghidra, browser, or OS coverage: not applicable (static JavaScript analysis). - Package or release metadata impact: none. - Security, privacy, process, or containment review: partial endpoint values are source expressions already present in the analyzed artifact; no new data leaves the analysis boundary. `sanitizeEndpointCandidate` continues to pass unparseable expressions through verbatim. ## Author readiness Human owner: @hemv-857 - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-31 01:09:35 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-01 21:45:12 +00:00

[PR #5810] [MERGED] fix(javascript): count one IPC observation per site-resolved channel #7075

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#5810

No description provided.

Rows
Columns