[GH-ISSUE #6641] [Bug] Next page discovery percent-decodes filesystem route segments and merges distinct pages #4022

New issue

Closed

opened 2026-10-31 00:46:08 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:46:08 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6641

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

The public analyze-javascript-application CLI maps both app/%61/page.js and app/a/page.js to /a, with two supporting sources. It silently omits app/100%/page.js. Both results report complete source parsing and candidate enumeration, with no route interpretation gaps. Next's source-filename normalization preserves ordinary percent sequences; its special %5F escape is handled separately. This concerns static source-path identity, not runtime reachability.

Reproduction

Run the following from the source checkout below with its built runtime: mise exec node@24.22.1 -- node /path/to/reproduce-percent.mjs. Save this script as /tmp/reproduce-percent.mjs. It creates synthetic Next App Router layouts/pages and isolated HOME/XDG directories, runs the real CLI, prints the relevant rows, and removes its own fixtures. No dependency installation, Next build, server, browser, or target execution is performed.

import {mkdtempSync,mkdirSync,writeFileSync,rmSync} from 'node:fs';
import {tmpdir} from 'node:os';
import {join,resolve} from 'node:path';
import {spawnSync} from 'node:child_process';
const entry=resolve('scripts/rea.mjs');
const scratch=mkdtempSync(join(tmpdir(),'rea-route-fixture-'));
try {
  for(const [name,segments] of [['reduced',['%61','a','%5Fpublic']],['malformed',['100%','normal']]]) {
    const app=join(scratch,name), home=join(scratch,`${name}-home`);
    mkdirSync(join(app,'app'),{recursive:true});mkdirSync(home);
    writeFileSync(join(app,'package.json'),JSON.stringify({name:'route-fixture',private:true,type:'module',dependencies:{next:'16.4.2',react:'19.2.0','react-dom':'19.2.0'}}));
    writeFileSync(join(app,'app/layout.js'),`import {createElement} from 'react';export default function Layout({children}){return createElement('html',null,createElement('body',null,children));}`);
    for(const segment of segments) {
      mkdirSync(join(app,'app',segment));
      writeFileSync(join(app,'app',segment,'page.js'),'export default function Page(){return null;}');
    }
    const env={PATH:process.env.PATH,HOME:home,XDG_CONFIG_HOME:join(home,'config'),XDG_DATA_HOME:join(home,'data'),XDG_CACHE_HOME:join(home,'cache'),XDG_STATE_HOME:join(home,'state'),NO_COLOR:'1'};
    const run=spawnSync(process.execPath,[entry,'analyze-javascript-application',app,'--format','json'],{cwd:home,env,encoding:'utf8',timeout:60000,maxBuffer:8*1024*1024});
    if(run.status!==0) throw new Error(`CLI failed: ${run.status} ${run.stderr}`);
    const inv=JSON.parse(run.stdout).normalized_result.route_inventory;
    const sources=new Map(inv.sources.map(s=>[s.id,s.key]));
    const routes=new Map(inv.routes.map(r=>[r.id,r.path]));
    console.log(JSON.stringify({case:name,exit:run.status,routes:inv.routes.map(r=>({path:r.path,source_count:r.source_count})),occurrences:inv.occurrences.map(o=>({source:sources.get(o.source_id),path:routes.get(o.route_id)})),coverage:inv.coverage,unknowns:inv.unknowns}));
  }
} finally { rmSync(scratch,{recursive:true,force:true}); }

The two reduced cases each failed on their initial CLI run and the script run. Every JavaScript file in the initial reduced fixtures also passed node --check. A prior broader fixture showed the same normalization problem for encoded punctuation: %40slot and %28promo%29 became /, and %5Bid%5D became /:id; those additional filenames are not needed for this reproduction.

Expected behavior

For filesystem source paths, retain separate static route candidates /%61 and /a; retain /100% rather than treating a filename as a malformed URL escape. Preserve the valid %5Fpublic control as /_public. Next's tagged filename-to-page helper preserves the path head, normalizeAppPath removes actual group/parallel segments, and AppPathnameNormalizer applies only %5F replacement. Those source-derived expectations do not assert that a Next server was run. If a filename cannot be interpreted, retain an explicit gap instead of silently declaring complete candidate enumeration.

Impact and scope

Distinct source pages merge into one exact route identity and supporting-source count, and another valid source filename disappears. The demonstrated inputs are bounded, self-authored Next App Router source directories. All CLI calls completed successfully; no target mutation, security effect, severe outage, or application-wide route completeness is claimed. Prevalence in real applications and published-release behavior are unknown.

REA version

Source main a281455f0b83c088ea30ebbb887398888ed167fb; package version 6.7.0, using its compiled runtime. The published npm release was not separately tested.

Runtime and platform

Node.js v24.22.1 through mise; Linux x64.

Target and artifact context

Self-authored JavaScript directories with a Next App Router root layout, page files, and a package manifest declaring Next 16.4.2 and React/React DOM 19.2.0. Dependencies were not installed in the target fixtures.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Actual stdout from the script, with both CLI exit statuses observed as zero:

{"case":"reduced","exit":0,"routes":[{"path":"/_public","source_count":1},{"path":"/a","source_count":2}],"occurrences":[{"source":"app/%5Fpublic/page.js","path":"/_public"},{"source":"app/%61/page.js","path":"/a"},{"source":"app/a/page.js","path":"/a"}],"coverage":{"scope":"provided-code-sources","source_parsing":"complete","candidate_enumeration":"complete","application_routes":"unknown"},"unknowns":[]}
{"case":"malformed","exit":0,"routes":[{"path":"/normal","source_count":1}],"occurrences":[{"source":"app/normal/page.js","path":"/normal"}],"coverage":{"scope":"provided-code-sources","source_parsing":"complete","candidate_enumeration":"complete","application_routes":"unknown"},"unknowns":[]}

Expected source: REA's documented Next page-path inventory. Next 16.4.2's filename-to-page conversion, path-tail removal, AppPathnameNormalizer, and UnderscoreNormalizer establish the source-derived mapping above.
Failure source: Executed public CLI and inline output above.
Evidence provenance: observed REA CLI output; expected Next mapping and causal explanation are source-derived.
Local verification: reproduced.
Reproduction completeness: complete for REA static analysis; a Next server/build and HTTP requests were not run.

Typed outcome or recovery expected

Preserve each source-derived exact route identity. Where interpretation is unavailable, expose that gap. Keep application_routes unknown and retain the correct escaped-underscore control.

Analysis or suspected cause

Source-derived: classifyNextAppRoutePath calls decodeURIComponent on every segment before recognizing groups, parallel slots, and dynamic parameters. A decode exception returns no fact. The inventory metadata path passes filesystem paths into this same classifier and creates a gap only for an explicit unknown result. These are causal breadcrumbs, not a prescribed implementation.

Additional context

Related open #6516 concerns attributing non-router paths to Next. This report uses actual app/ roots and concerns interpretation of their segment bytes. Last known good revision: NOT_ENOUGH_INFO; no regression boundary established.

Restoration check, proposed and not run after drafting: the reduced case should retain separate /%61, /a, and /_public candidates, each with one source; the percent-sign case should retain /100% and /normal, or an explicit supported interpretation gap for the former. Do not hide the merge by removing the valid a or underscore controls.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: Executed static directory analysis merges distinct route identities and silently drops a source-path candidate for bounded supported inputs; source-count evidence is consequently wrong.
Workaround status: unverified
Workaround basis: No practical result-preserving workaround through REA was exercised; manually inspecting filenames does not establish corrected retained route Evidence.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6641 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior The public `analyze-javascript-application` CLI maps both `app/%61/page.js` and `app/a/page.js` to `/a`, with two supporting sources. It silently omits `app/100%/page.js`. Both results report complete source parsing and candidate enumeration, with no route interpretation gaps. Next's source-filename normalization preserves ordinary percent sequences; its special `%5F` escape is handled separately. This concerns static source-path identity, not runtime reachability. ### Reproduction Run the following from the source checkout below with its built runtime: `mise exec node@24.22.1 -- node /path/to/reproduce-percent.mjs`. Save this script as `/tmp/reproduce-percent.mjs`. It creates synthetic Next App Router layouts/pages and isolated HOME/XDG directories, runs the real CLI, prints the relevant rows, and removes its own fixtures. No dependency installation, Next build, server, browser, or target execution is performed. ```js import {mkdtempSync,mkdirSync,writeFileSync,rmSync} from 'node:fs'; import {tmpdir} from 'node:os'; import {join,resolve} from 'node:path'; import {spawnSync} from 'node:child_process'; const entry=resolve('scripts/rea.mjs'); const scratch=mkdtempSync(join(tmpdir(),'rea-route-fixture-')); try { for(const [name,segments] of [['reduced',['%61','a','%5Fpublic']],['malformed',['100%','normal']]]) { const app=join(scratch,name), home=join(scratch,`${name}-home`); mkdirSync(join(app,'app'),{recursive:true});mkdirSync(home); writeFileSync(join(app,'package.json'),JSON.stringify({name:'route-fixture',private:true,type:'module',dependencies:{next:'16.4.2',react:'19.2.0','react-dom':'19.2.0'}})); writeFileSync(join(app,'app/layout.js'),`import {createElement} from 'react';export default function Layout({children}){return createElement('html',null,createElement('body',null,children));}`); for(const segment of segments) { mkdirSync(join(app,'app',segment)); writeFileSync(join(app,'app',segment,'page.js'),'export default function Page(){return null;}'); } const env={PATH:process.env.PATH,HOME:home,XDG_CONFIG_HOME:join(home,'config'),XDG_DATA_HOME:join(home,'data'),XDG_CACHE_HOME:join(home,'cache'),XDG_STATE_HOME:join(home,'state'),NO_COLOR:'1'}; const run=spawnSync(process.execPath,[entry,'analyze-javascript-application',app,'--format','json'],{cwd:home,env,encoding:'utf8',timeout:60000,maxBuffer:8*1024*1024}); if(run.status!==0) throw new Error(`CLI failed: ${run.status} ${run.stderr}`); const inv=JSON.parse(run.stdout).normalized_result.route_inventory; const sources=new Map(inv.sources.map(s=>[s.id,s.key])); const routes=new Map(inv.routes.map(r=>[r.id,r.path])); console.log(JSON.stringify({case:name,exit:run.status,routes:inv.routes.map(r=>({path:r.path,source_count:r.source_count})),occurrences:inv.occurrences.map(o=>({source:sources.get(o.source_id),path:routes.get(o.route_id)})),coverage:inv.coverage,unknowns:inv.unknowns})); } } finally { rmSync(scratch,{recursive:true,force:true}); } ``` The two reduced cases each failed on their initial CLI run and the script run. Every JavaScript file in the initial reduced fixtures also passed `node --check`. A prior broader fixture showed the same normalization problem for encoded punctuation: `%40slot` and `%28promo%29` became `/`, and `%5Bid%5D` became `/:id`; those additional filenames are not needed for this reproduction. ### Expected behavior For filesystem source paths, retain separate static route candidates `/%61` and `/a`; retain `/100%` rather than treating a filename as a malformed URL escape. Preserve the valid `%5Fpublic` control as `/_public`. Next's tagged filename-to-page helper preserves the path head, `normalizeAppPath` removes actual group/parallel segments, and `AppPathnameNormalizer` applies only `%5F` replacement. Those source-derived expectations do not assert that a Next server was run. If a filename cannot be interpreted, retain an explicit gap instead of silently declaring complete candidate enumeration. ### Impact and scope Distinct source pages merge into one exact route identity and supporting-source count, and another valid source filename disappears. The demonstrated inputs are bounded, self-authored Next App Router source directories. All CLI calls completed successfully; no target mutation, security effect, severe outage, or application-wide route completeness is claimed. Prevalence in real applications and published-release behavior are unknown. ### REA version Source main `a281455f0b83c088ea30ebbb887398888ed167fb`; package version `6.7.0`, using its compiled runtime. The published npm release was not separately tested. ### Runtime and platform Node.js v24.22.1 through mise; Linux x64. ### Target and artifact context Self-authored JavaScript directories with a Next App Router root layout, page files, and a package manifest declaring Next 16.4.2 and React/React DOM 19.2.0. Dependencies were not installed in the target fixtures. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Actual stdout from the script, with both CLI exit statuses observed as zero: ```jsonl {"case":"reduced","exit":0,"routes":[{"path":"/_public","source_count":1},{"path":"/a","source_count":2}],"occurrences":[{"source":"app/%5Fpublic/page.js","path":"/_public"},{"source":"app/%61/page.js","path":"/a"},{"source":"app/a/page.js","path":"/a"}],"coverage":{"scope":"provided-code-sources","source_parsing":"complete","candidate_enumeration":"complete","application_routes":"unknown"},"unknowns":[]} {"case":"malformed","exit":0,"routes":[{"path":"/normal","source_count":1}],"occurrences":[{"source":"app/normal/page.js","path":"/normal"}],"coverage":{"scope":"provided-code-sources","source_parsing":"complete","candidate_enumeration":"complete","application_routes":"unknown"},"unknowns":[]} ``` - Expected source: REA's [documented Next page-path inventory](https://github.com/morluto/rea/blob/a281455f0b83c088ea30ebbb887398888ed167fb/docs/javascript-route-inventory.md#L60-L86). Next 16.4.2's [filename-to-page conversion](https://github.com/vercel/next.js/blob/v16.4.2/packages/next/src/shared/lib/page-path/absolute-path-to-page.ts#L30-L38), [path-tail removal](https://github.com/vercel/next.js/blob/v16.4.2/packages/next/src/shared/lib/page-path/remove-page-path-tail.ts#L12-L28), [AppPathnameNormalizer](https://github.com/vercel/next.js/blob/v16.4.2/packages/next/src/server/normalizers/built/app/app-pathname-normalizer.ts#L5-L18), and [UnderscoreNormalizer](https://github.com/vercel/next.js/blob/v16.4.2/packages/next/src/server/normalizers/underscore-normalizer.ts#L5-L8) establish the source-derived mapping above. - Failure source: Executed public CLI and inline output above. - Evidence provenance: observed REA CLI output; expected Next mapping and causal explanation are source-derived. - Local verification: reproduced. - Reproduction completeness: complete for REA static analysis; a Next server/build and HTTP requests were not run. ### Typed outcome or recovery expected Preserve each source-derived exact route identity. Where interpretation is unavailable, expose that gap. Keep `application_routes` unknown and retain the correct escaped-underscore control. ### Analysis or suspected cause Source-derived: [classifyNextAppRoutePath](https://github.com/morluto/rea/blob/a281455f0b83c088ea30ebbb887398888ed167fb/src/domain/javascript/javascriptRoutePaths.ts#L43-L78) calls `decodeURIComponent` on every segment before recognizing groups, parallel slots, and dynamic parameters. A decode exception returns no fact. [The inventory metadata path](https://github.com/morluto/rea/blob/a281455f0b83c088ea30ebbb887398888ed167fb/src/domain/javascript/javascriptRouteInventory.ts#L458-L464) passes filesystem paths into this same classifier and creates a gap only for an explicit unknown result. These are causal breadcrumbs, not a prescribed implementation. ### Additional context Related open [#6516](https://github.com/morluto/rea/issues/6516) concerns attributing non-router paths to Next. This report uses actual `app/` roots and concerns interpretation of their segment bytes. Last known good revision: NOT_ENOUGH_INFO; no regression boundary established. Restoration check, proposed and not run after drafting: the reduced case should retain separate `/%61`, `/a`, and `/_public` candidates, each with one source; the percent-sign case should retain `/100%` and `/normal`, or an explicit supported interpretation gap for the former. Do not hide the merge by removing the valid `a` or underscore controls. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: Executed static directory analysis merges distinct route identities and silently drops a source-path candidate for bounded supported inputs; source-count evidence is consequently wrong. - Workaround status: unverified - Workaround basis: No practical result-preserving workaround through REA was exercised; manually inspecting filenames does not establish corrected retained route Evidence.

bsakaguchi closed this issue

2026-10-31 00:46:08 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:03 +00:00

[PR #4022] [CLOSED] POC: Systems bridge for yoimuya #5996