[GH-ISSUE #3539] [Feature] analyze_web_bundle: surface path- and URL-shaped literals passed to project request wrappers #1315

New issue

Open

opened 2026-10-16 21:04:09 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-16 21:04:09 +00:00

Owner

Originally created by @Rinkia on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3539

Before submitting

I searched the open and closed issues and pull requests for a related proposal. Related: #2806 (false-positive endpoints, fixed by demoting unverified receivers to low-confidence candidates). This is the opposite gap: real endpoints that aren't surfaced at all.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

endpointArgument (src/domain/javascript/javascriptStaticAnalysisHelpers.ts:594) recognises a request URL only when the callee is fetch/.fetch, WebSocket, EventSource, .open, or is named get/post/put/patch/delete/request. Real apps usually call their API through a project wrapper (apiPostWithRetry(path, body), api.call(path)), or keep paths in constant arrays. Those literals aren't reported at all, not even as low-confidence candidates. Meanwhile, any string passed to a method named get on an unknown receiver becomes a candidate, so the candidate list shows non-paths while the real API surface is missing.

Minimal fixture (served locally, analysed with analyze-web-bundle, rea-agents@6.5.0):

async function apiPostWithRetry(path, body) { const url = BASE + path; return fetch(url, { method: "POST", body: JSON.stringify(body) }); }
const BASE = "https://api.example.test";
const AUTH_ROUTES = ["/api/daily/stats", "/api/progress/complete", "/api/multiplayer/hint"];
function report() { return fetch("/api/feedback", { method: "POST" }); }
function complete(id, ms) { return apiPostWithRetry("/api/daily/complete", { puzzleId: id, completionTimeMs: ms }); }
const settings = new Map(); settings.get("dpl");
const http = { post: (u) => fetch(u) }; http.post("/api/verified-post");
window.app = { report, complete, AUTH_ROUTES };

Result (completeness: complete, 1 script, 0 parse failures):

Literal	Reported
`/api/feedback` (`fetch`)	✅ endpoint, `call:fetch`
`/api/verified-post` (`http.post`)	✅ endpoint, `call:http.post`
`/api/daily/complete` (first argument of `apiPostWithRetry`)	❌ not reported
`/api/daily/stats`, `/api/progress/complete`, `/api/multiplayer/hint` (array)	❌ not reported
`https://api.example.test` (base URL constant)	❌ not reported
`dpl` (`settings.get`, where `settings = new Map()`)	⚠️ `endpoint_candidate`, `call:settings.get:unverified-network-receiver`

Real-world shape: on a public Next.js game site, analyze-web-bundle parsed 37 first-party scripts completely and reported 1 endpoint plus low-confidence candidates dpl, date, size, [], [...], [[...]]. The same scripts (via export-web-scripts) contain about 25 distinct /api/... literals, passed through a request wrapper or kept in arrays. The network capture confirmed several of them at runtime.

Use cases

API-surface discovery from shipped bundles, the main reason to run analyze_web_bundle on a SPA.
Pairing static candidates with runtime network captures (capture_browser_scenario), which already observe these endpoints.

Desired outcome and success criteria

Path-shaped (/segment…) and absolute http(s):// / ws(s):// string literals become endpoint_candidate inferences (low confidence, a distinct detector such as literal:path-shaped) when they're the first argument of any call or an element of a string array. They stay separate from verified endpoints.
Optionally, drop or down-rank candidates whose value isn't URL- or path-shaped (dpl, size, []), or skip .get on receivers bound to new Map() / new URLSearchParams() through a same-scope const, extending isKeyedCollectionReceiver, which currently checks only the spelled-out new Map().get(…) form.
Success: on the fixture, the four wrapper/array paths and the base URL appear as candidates, and dpl doesn't.

Expected interaction with targets

Read existing files or captured data.

Verification available so far

I have a prototype or observed results.

Verification details

Fixture above, served by a local Node HTTP server, loaded in headless Chromium, then rea analyze-web-bundle http://127.0.0.1:9222 <target> --observation-ms 1500. Output as tabulated. Source reading on main @ eb44921c4.

Alternatives considered

Treating every string literal as a candidate: too noisy, the problem #2806 fixed. Restricting to path- or URL-shaped literals keeps noise low.
Resolving wrapper functions interprocedurally (apiPostWithRetry → fetch(BASE + path)). More precise but much larger. The shape heuristic is a cheap first step.

Possible direction

Add a path- and URL-shape predicate (leading / followed by a segment, or an absolute URL), and apply it to string literals in call arguments and array elements, emitting endpoint_candidate with its own detector name.

References

#2806, #2805, #2841
src/domain/javascript/javascriptStaticAnalysisHelpers.ts (endpointArgument, isKeyedCollectionReceiver)
src/domain/webBundleAnalyzerInspection.ts (addEndpointCandidate)

Originally created by @Rinkia on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3539 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. Related: #2806 (false-positive endpoints, fixed by demoting unverified receivers to low-confidence candidates). This is the opposite gap: real endpoints that aren't surfaced at all. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation `endpointArgument` (`src/domain/javascript/javascriptStaticAnalysisHelpers.ts:594`) recognises a request URL only when the callee is `fetch`/`.fetch`, `WebSocket`, `EventSource`, `.open`, or is named `get`/`post`/`put`/`patch`/`delete`/`request`. Real apps usually call their API through a project wrapper (`apiPostWithRetry(path, body)`, `api.call(path)`), or keep paths in constant arrays. Those literals aren't reported at all, not even as low-confidence candidates. Meanwhile, any string passed to a method named `get` on an unknown receiver becomes a candidate, so the candidate list shows non-paths while the real API surface is missing. Minimal fixture (served locally, analysed with `analyze-web-bundle`, `rea-agents@6.5.0`): ```js async function apiPostWithRetry(path, body) { const url = BASE + path; return fetch(url, { method: "POST", body: JSON.stringify(body) }); } const BASE = "https://api.example.test"; const AUTH_ROUTES = ["/api/daily/stats", "/api/progress/complete", "/api/multiplayer/hint"]; function report() { return fetch("/api/feedback", { method: "POST" }); } function complete(id, ms) { return apiPostWithRetry("/api/daily/complete", { puzzleId: id, completionTimeMs: ms }); } const settings = new Map(); settings.get("dpl"); const http = { post: (u) => fetch(u) }; http.post("/api/verified-post"); window.app = { report, complete, AUTH_ROUTES }; ``` Result (`completeness: complete`, 1 script, 0 parse failures): | Literal | Reported | |---|---| | `/api/feedback` (`fetch`) | ✅ endpoint, `call:fetch` | | `/api/verified-post` (`http.post`) | ✅ endpoint, `call:http.post` | | `/api/daily/complete` (first argument of `apiPostWithRetry`) | ❌ not reported | | `/api/daily/stats`, `/api/progress/complete`, `/api/multiplayer/hint` (array) | ❌ not reported | | `https://api.example.test` (base URL constant) | ❌ not reported | | `dpl` (`settings.get`, where `settings = new Map()`) | ⚠️ `endpoint_candidate`, `call:settings.get:unverified-network-receiver` | Real-world shape: on a public Next.js game site, `analyze-web-bundle` parsed 37 first-party scripts completely and reported 1 endpoint plus low-confidence candidates `dpl`, `date`, `size`, `[]`, `[...]`, `[[...]]`. The same scripts (via `export-web-scripts`) contain about 25 distinct `/api/...` literals, passed through a request wrapper or kept in arrays. The network capture confirmed several of them at runtime. ### Use cases - API-surface discovery from shipped bundles, the main reason to run `analyze_web_bundle` on a SPA. - Pairing static candidates with runtime network captures (`capture_browser_scenario`), which already observe these endpoints. ### Desired outcome and success criteria - Path-shaped (`/segment…`) and absolute `http(s)://` / `ws(s)://` string literals become `endpoint_candidate` inferences (low confidence, a distinct detector such as `literal:path-shaped`) when they're the first argument of any call or an element of a string array. They stay separate from verified endpoints. - Optionally, drop or down-rank candidates whose value isn't URL- or path-shaped (`dpl`, `size`, `[]`), or skip `.get` on receivers bound to `new Map()` / `new URLSearchParams()` through a same-scope `const`, extending `isKeyedCollectionReceiver`, which currently checks only the spelled-out `new Map().get(…)` form. - Success: on the fixture, the four wrapper/array paths and the base URL appear as candidates, and `dpl` doesn't. ### Expected interaction with targets Read existing files or captured data. ### Verification available so far I have a prototype or observed results. ### Verification details Fixture above, served by a local Node HTTP server, loaded in headless Chromium, then `rea analyze-web-bundle http://127.0.0.1:9222 <target> --observation-ms 1500`. Output as tabulated. Source reading on `main` @ `eb44921c4`. ### Alternatives considered - Treating every string literal as a candidate: too noisy, the problem #2806 fixed. Restricting to path- or URL-shaped literals keeps noise low. - Resolving wrapper functions interprocedurally (`apiPostWithRetry` → `fetch(BASE + path)`). More precise but much larger. The shape heuristic is a cheap first step. ### Possible direction Add a path- and URL-shape predicate (leading `/` followed by a segment, or an absolute URL), and apply it to string literals in call arguments and array elements, emitting `endpoint_candidate` with its own detector name. ### References - #2806, #2805, #2841 - `src/domain/javascript/javascriptStaticAnalysisHelpers.ts` (`endpointArgument`, `isKeyedCollectionReceiver`) - `src/domain/webBundleAnalyzerInspection.ts` (`addEndpointCandidate`)

bsakaguchi referenced this issue

2026-10-17 21:01:32 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

bsakaguchi referenced this issue

2026-10-17 21:10:21 +00:00

[PR #1315] [CLOSED] fix(symbols): discover emitted Swift protocol descriptors #2795

bsakaguchi referenced this issue

2026-10-31 00:39:50 +00:00

[GH-ISSUE #5580] [Bug] get_objc_protocols counts Objective-C method labels containing _TtP as protocols #3390