morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4772] [Feature] Static route inventory for web bundles: whole paths, typed route trees, Next.js App Router routes, and a route coverage count #2481

New issue

Closed

opened 2026-10-17 21:05:26 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-17 21:05:26 +00:00

Owner

Originally created by @JoshOsullivan-au on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4772

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

When an analyst maps a signed-in web app, the first question is "what screens does this app ship?". analyze_web_bundle and analyze_javascript_application answer it only for routes written as a literal path or route string property, or as the first string argument of route, addRoute, useRoutes, createBrowserRouter or createHashRouter (inspectRouteProperty and the router-call branch in src/domain/webBundleAnalyzerInspection.ts for analyze_web_bundle; inspectRouteProperty and STATIC_ROUTE_CALL_NAMES in src/domain/javascript/javascriptStaticAnalysisCalls.ts and javascriptStaticAnalysisHelpers.ts for analyze_javascript_application; checked on main at 3fc0075, 17/10/2026). Next.js appears only as a vendor fingerprint (__NEXT_DATA__, /_next/).

Three common shapes fall outside that, so the route candidates come back nearly empty and an agent concludes the app is small:

Typed route trees. Route libraries that declare a nested object and derive paths from it, for example ROUTES = defineRoutes({ reports: { _$: "reports", settings: { _$: "settings" } }, ... }). Segments sit under a non-path key, and full paths only exist after joining parent and child.
Next.js App Router. Routes come from the file system and appear in chunk names (static/chunks/app/<route>/page-<hash>.js), the webpack runtime chunk map, or, under Turbopack, only as route strings ("/app/profile/:profile_id/competitors") and base-path-relative navigation targets (push("/planner") under an /app base path).
Nested segments. Even when segments are found, they are reported one by one (settings), not as the path the app serves (/reports/settings).

There is also no route inventory as such: route candidates share the flat endpoint list with network calls, so nothing reports how many routes exist or how many a capture covered.

Use cases

Two production web apps, observed read-only in the analyst's own signed-in accounts (17/10/2026). App A is a Vite + React SPA with a typed route tree plus a Next.js (Turbopack) sub-app under a base path. App B is a Next.js (Turbopack) app. Bundles were read from the static chunks the pages had already loaded; no discovered route was requested for discovery.

	App A	App B
Page routes the app ships (prototype inventory, noise filtered)	90	62
Found by the app's own navigation links	12	19
Found by REA's current static route detection (6.3.0)	1 (`/`)	0
Found by `analyze_web_bundle` on 6.5.0 (17/10/2026; two App A pages, one App B page, all scripts parsed)	1 (`/`) of 22 candidates	0 of 4 candidates
REA literal candidates that were not routes	``, `/`, `/lp/`, `/elements/`, `/state/items/0`, `contact_email`, `phone`	SVG path data (`M16.0123 18.7087H0V34...`), `:`
Where the real routes came from	typed route tree (110 nodes; 81 page routes), base-path navigation (7), navigation (12)	route string constants (60), navigation (19)
Screens observed in a navigation-first walk	24 of 90 (27%)	25 of 62 (40%)
Screens observed after inventory-driven capture	70 of 90 (78%)	43 of 62 (69%)

The capture step is a local prototype outside REA (Playwright over CDP, navigation only, system routes excluded). It is included only to show that the missing routes are real, reachable screens. Remaining gaps were id routes with no example link (17 and 8) and screens that answered 404 or a permission page (1 and 14).

Environment: REA via rea-agents 6.3.0 for the original run, re-checked on 6.5.0 (analyze_web_bundle, same result); source checked on main at 3fc0075. macOS, Chrome 155 (dedicated profile, --remote-debugging-port, --headless=new).

Desired outcome and success criteria

A route inventory in the static JavaScript evidence, still with no execution and no network effect:

Whole paths. Join nested route objects (children, typed trees) into full paths, keep the per-segment origin, and preserve :param and wildcard markers.
Typed route trees. Recognise nested object literals whose nodes carry a segment under a configurable or detected key (for example _$), including nodes with no explicit segment, reported as inferred.
Next.js App Router. Read route names from app/<route>/page chunk names and runtime chunk maps, strip route groups (group), map [param] and [...param], and detect a base path from <base>/_next/ script URLs.
Route string constants and navigation targets. Absolute strings under the app's own top-level segments and arguments of href, push, replace, prefetch, navigate, with a lower confidence than structural sources.
A route inventory section separate from network endpoints: each route with its mechanisms, a page / system (auth, billing, callbacks) / unknown class, noise excluded (JSON pointers, SVG path data), and a count, so a later capture can report "N of M routes observed".

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Static analysis of already-captured scripts only. No route is requested to test existence; no new network, execution, injection or mutation effect.
Not a crawler, not a sitemap fetcher. (An optional, separately-scoped idea: accept a caller-supplied sitemap.xml file as an extra inventory source, mirroring how fetch_source_maps stays opt-in.)
No attempt to defeat obfuscation beyond what the existing static analysis already does.

Questions for maintainers:

Should the route inventory be a section of the existing Evidence for analyze_web_bundle / analyze_javascript_application, or a separate operation?
Is a configurable segment key for typed route trees acceptable, or should detection stay heuristic-only?
Would a caller-supplied sitemap file as an inventory source need its own scope decision?

Verification available so far

I have a prototype or observed results

Verification details (optional)

Self-authored fixtures only (no third-party bundles):

A Vite + React Router build with nested children routes.
A typed route tree fixture (defineRoutes-style, with and without explicit segments).
A Next.js App Router build (webpack and Turbopack) with route groups, [param], [...param] and a basePath.
Negative fixtures: JSON-pointer path properties, SVG d strings, path keys in non-router config.

Checks: focused tests beside javascriptStaticAnalysis.routeProperties.test.ts, npm run check:fast, npm run docs:check, and the browser-observation real-provider lane for analyze_web_bundle.

Alternatives considered

The current detector is correct for what it targets but sees almost none of these shapes. The analyst has to download chunks and grep by hand, as we did, which loses REA's Evidence provenance, locations and coverage reporting.

References

#3608 (React semantic reconstruction RFC) lists router discovery as one of six layers: React Router calls, <Route path>, Next.js pages/** and app/**/page.js source paths, TanStack Router and Remix, as React graph nodes. This proposal is narrower and works on production web bundles, where those source paths are gone: Turbopack chunk names are hashed, so routes survive only as strings, typed route trees and base-path navigation. It adds what #3608 does not cover: typed route trees, joining nested segments into whole paths, base paths, a noise filter, and a route inventory with a count for coverage. If accepted, its route records could feed #3608's react-route nodes rather than compete with them.
#3539 surfaces path-shaped literals passed to request wrappers as endpoint candidates. Item 4 of the desired outcome uses a similar shape test, but for page routes, kept apart from endpoints. The two could share one path-shape predicate.

Authored by Josh O'Sullivan, 17.10.2026

Originally created by @JoshOsullivan-au on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4772 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation When an analyst maps a signed-in web app, the first question is "what screens does this app ship?". `analyze_web_bundle` and `analyze_javascript_application` answer it only for routes written as a literal `path` or `route` string property, or as the first string argument of `route`, `addRoute`, `useRoutes`, `createBrowserRouter` or `createHashRouter` (`inspectRouteProperty` and the router-call branch in `src/domain/webBundleAnalyzerInspection.ts` for `analyze_web_bundle`; `inspectRouteProperty` and `STATIC_ROUTE_CALL_NAMES` in `src/domain/javascript/javascriptStaticAnalysisCalls.ts` and `javascriptStaticAnalysisHelpers.ts` for `analyze_javascript_application`; checked on `main` at `3fc0075`, 17/10/2026). Next.js appears only as a vendor fingerprint (`__NEXT_DATA__`, `/_next/`). Three common shapes fall outside that, so the route candidates come back nearly empty and an agent concludes the app is small: 1. **Typed route trees.** Route libraries that declare a nested object and derive paths from it, for example `ROUTES = defineRoutes({ reports: { _$: "reports", settings: { _$: "settings" } }, ... })`. Segments sit under a non-`path` key, and full paths only exist after joining parent and child. 2. **Next.js App Router.** Routes come from the file system and appear in chunk names (`static/chunks/app/<route>/page-<hash>.js`), the webpack runtime chunk map, or, under Turbopack, only as route strings (`"/app/profile/:profile_id/competitors"`) and base-path-relative navigation targets (`push("/planner")` under an `/app` base path). 3. **Nested segments.** Even when segments are found, they are reported one by one (`settings`), not as the path the app serves (`/reports/settings`). There is also no route inventory as such: route candidates share the flat endpoint list with network calls, so nothing reports how many routes exist or how many a capture covered. ### Use cases Two production web apps, observed read-only in the analyst's own signed-in accounts (17/10/2026). App A is a Vite + React SPA with a typed route tree plus a Next.js (Turbopack) sub-app under a base path. App B is a Next.js (Turbopack) app. Bundles were read from the static chunks the pages had already loaded; no discovered route was requested for discovery. | | App A | App B | | --- | --- | --- | | Page routes the app ships (prototype inventory, noise filtered) | 90 | 62 | | Found by the app's own navigation links | 12 | 19 | | Found by REA's current static route detection (6.3.0) | 1 (`/`) | 0 | | Found by `analyze_web_bundle` on 6.5.0 (17/10/2026; two App A pages, one App B page, all scripts parsed) | 1 (`/`) of 22 candidates | 0 of 4 candidates | | REA literal candidates that were not routes | `*`, `/*`, `/lp/*`, `/elements/*`, `/state/items/0`, `contact_email`, `phone` | SVG path data (`M16.0123 18.7087H0V34...`), `:` | | Where the real routes came from | typed route tree (110 nodes; 81 page routes), base-path navigation (7), navigation (12) | route string constants (60), navigation (19) | | Screens observed in a navigation-first walk | 24 of 90 (27%) | 25 of 62 (40%) | | Screens observed after inventory-driven capture | 70 of 90 (78%) | 43 of 62 (69%) | The capture step is a local prototype outside REA (Playwright over CDP, navigation only, system routes excluded). It is included only to show that the missing routes are real, reachable screens. Remaining gaps were id routes with no example link (17 and 8) and screens that answered 404 or a permission page (1 and 14). Environment: REA via `rea-agents` 6.3.0 for the original run, re-checked on 6.5.0 (`analyze_web_bundle`, same result); source checked on `main` at `3fc0075`. macOS, Chrome 155 (dedicated profile, `--remote-debugging-port`, `--headless=new`). ### Desired outcome and success criteria A route inventory in the static JavaScript evidence, still with no execution and no network effect: 1. **Whole paths.** Join nested route objects (`children`, typed trees) into full paths, keep the per-segment origin, and preserve `:param` and wildcard markers. 2. **Typed route trees.** Recognise nested object literals whose nodes carry a segment under a configurable or detected key (for example `_$`), including nodes with no explicit segment, reported as inferred. 3. **Next.js App Router.** Read route names from `app/<route>/page` chunk names and runtime chunk maps, strip route groups `(group)`, map `[param]` and `[...param]`, and detect a base path from `<base>/_next/` script URLs. 4. **Route string constants and navigation targets.** Absolute strings under the app's own top-level segments and arguments of `href`, `push`, `replace`, `prefetch`, `navigate`, with a lower confidence than structural sources. 5. **A route inventory section** separate from network endpoints: each route with its mechanisms, a page / system (auth, billing, callbacks) / unknown class, noise excluded (JSON pointers, SVG path data), and a count, so a later capture can report "N of M routes observed". ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) - Static analysis of already-captured scripts only. No route is requested to test existence; no new network, execution, injection or mutation effect. - Not a crawler, not a sitemap fetcher. (An optional, separately-scoped idea: accept a caller-supplied `sitemap.xml` file as an extra inventory source, mirroring how `fetch_source_maps` stays opt-in.) - No attempt to defeat obfuscation beyond what the existing static analysis already does. Questions for maintainers: 1. Should the route inventory be a section of the existing Evidence for `analyze_web_bundle` / `analyze_javascript_application`, or a separate operation? 2. Is a configurable segment key for typed route trees acceptable, or should detection stay heuristic-only? 3. Would a caller-supplied sitemap file as an inventory source need its own scope decision? ### Verification available so far I have a prototype or observed results ### Verification details (optional) Self-authored fixtures only (no third-party bundles): - A Vite + React Router build with nested `children` routes. - A typed route tree fixture (`defineRoutes`-style, with and without explicit segments). - A Next.js App Router build (webpack and Turbopack) with route groups, `[param]`, `[...param]` and a `basePath`. - Negative fixtures: JSON-pointer `path` properties, SVG `d` strings, `path` keys in non-router config. Checks: focused tests beside `javascriptStaticAnalysis.routeProperties.test.ts`, `npm run check:fast`, `npm run docs:check`, and the browser-observation real-provider lane for `analyze_web_bundle`. ### Alternatives considered The current detector is correct for what it targets but sees almost none of these shapes. The analyst has to download chunks and grep by hand, as we did, which loses REA's Evidence provenance, locations and coverage reporting. ### References - #3608 (React semantic reconstruction RFC) lists router discovery as one of six layers: React Router calls, `<Route path>`, Next.js `pages/**` and `app/**/page.js` source paths, TanStack Router and Remix, as React graph nodes. This proposal is narrower and works on production web bundles, where those source paths are gone: Turbopack chunk names are hashed, so routes survive only as strings, typed route trees and base-path navigation. It adds what #3608 does not cover: typed route trees, joining nested segments into whole paths, base paths, a noise filter, and a route inventory with a count for coverage. If accepted, its route records could feed #3608's `react-route` nodes rather than compete with them. - #3539 surfaces path-shaped literals passed to request wrappers as endpoint candidates. Item 4 of the desired outcome uses a similar shape test, but for page routes, kept apart from endpoints. The two could share one path-shape predicate. --- Authored by Josh O'Sullivan, 17.10.2026

bsakaguchi commented

2026-10-17 21:05:28 +00:00

Author

Owner

@tazheeb-shamsi commented on GitHub (Oct 17, 2026):

Hi @JoshOsullivan-au ! I’d like to work on this feature. Before I start implementing, could maintainers help settle the open design questions?

Should the route inventory extend the existing analyze_web_bundle / analyze_javascript_application Evidence, or should it be a separate operation? My preference is to extend the existing analysis flows if that fits their current contracts.
For typed route trees, should segment keys such as _$ be configurable, or should detection remain heuristic-only?
I’ll keep caller-supplied sitemap input out of scope unless it’s approved separately.

I’ll keep the implementation static-only over already-captured scripts: no crawling, route probing, execution, or network requests. I’ll also keep route records distinct from network endpoints and include source/confidence information as the proposal describes.

Please let me know which contract and scope you’d prefer before I begin wiring this into the shared workflows.

@tazheeb-shamsi commented on GitHub (Oct 17, 2026): Hi @JoshOsullivan-au ! I’d like to work on this feature. Before I start implementing, could maintainers help settle the open design questions? - Should the route inventory extend the existing `analyze_web_bundle` / `analyze_javascript_application` Evidence, or should it be a separate operation? My preference is to extend the existing analysis flows if that fits their current contracts. - For typed route trees, should segment keys such as `_$` be configurable, or should detection remain heuristic-only? - I’ll keep caller-supplied sitemap input out of scope unless it’s approved separately. I’ll keep the implementation static-only over already-captured scripts: no crawling, route probing, execution, or network requests. I’ll also keep route records distinct from network endpoints and include source/confidence information as the proposal describes. Please let me know which contract and scope you’d prefer before I begin wiring this into the shared workflows.

bsakaguchi closed this issue

2026-10-31 00:34:24 +00:00

bsakaguchi commented

2026-10-31 00:34:26 +00:00

Author

Owner

@JoshOsullivan-au commented on GitHub (Oct 18, 2026):

Thanks @tazheeb-shamsi, happy for you to take this on. Once there's a branch, I can run it on the two apps from the issue and notify you of the outcome.

@JoshOsullivan-au commented on GitHub (Oct 18, 2026): Thanks @tazheeb-shamsi, happy for you to take this on. Once there's a branch, I can run it on the two apps from the issue and notify you of the outcome.

bsakaguchi commented

2026-10-31 00:34:26 +00:00

Author

Owner

@XinwangTao commented on GitHub (Oct 18, 2026):

The gap is real and the benchmark numbers make it concrete — 1 of 22 candidates on App A means current literal-based detection is effectively blind on typed route trees and Next.js chunk naming. Two design notes: (1) for the typed route tree case, emitting the joined full paths (not just leaf segments) is what makes the inventory useful for coverage questions, so the tree walk should produce parent-joined outputs by default; (2) for Next.js, chunk-name parsing and the runtime route string map are complementary signals — chunk names give the file-system layout while route strings give the served paths — so reporting both with a source tag on each candidate would let analysts distinguish 'definitely served' from 'probably bundled'. A route coverage count only makes sense if the inventory is anchored to something stable (e.g. a root path or the navigation link set), otherwise the denominator drifts between captures.

@XinwangTao commented on GitHub (Oct 18, 2026): The gap is real and the benchmark numbers make it concrete — 1 of 22 candidates on App A means current literal-based detection is effectively blind on typed route trees and Next.js chunk naming. Two design notes: (1) for the typed route tree case, emitting the joined full paths (not just leaf segments) is what makes the inventory useful for coverage questions, so the tree walk should produce parent-joined outputs by default; (2) for Next.js, chunk-name parsing and the runtime route string map are complementary signals — chunk names give the file-system layout while route strings give the served paths — so reporting both with a source tag on each candidate would let analysts distinguish 'definitely served' from 'probably bundled'. A route coverage count only makes sense if the inventory is anchored to something stable (e.g. a root path or the navigation link set), otherwise the denominator drifts between captures.

bsakaguchi referenced this issue

2026-10-31 01:05:16 +00:00

[PR #2481] [MERGED] fix(setup): pin Hopper's macOS publisher identity before install #5263

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2481

No description provided.

Rows
Columns