morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7633] [Bug] VitePress-rendered artifact Markdown substitutes an emoji for a literal entry name #4558

New issue

Closed

opened 2026-10-31 00:50:46 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-31 00:50:46 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7633

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

CLI

Observed behavior

For a ZIP named :) with an entry also named :), node scripts/rea.mjs inspect-artifact "$WORK/:)" --format md succeeds. Rendering its output with the checkout's locked VitePress renderer displays 😃 as the artifact name under subject.name. The nested inventory JSON table cell also displays "logical_path":"😃" for the entry.

The emitted Markdown and paired JSON retain :). Substitution first occurs in the renderer, whose emoji plugin is enabled by default. A second renderer instance with empty emoji definitions/shortcuts preserves :) in both positions. Two independently executed CLI/renderer comparisons reproduced the substitution.

Reproduction

Use the already-built baseline checkout with its locked dependencies installed and Node 24.22.1 available as node. Run the following from its root. It generates the ZIP inline, isolates HOME/XDG paths, compares the rendered output, and deletes its temporary files on exit:

export WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
mkdir -p "$WORK/home" "$WORK/config" "$WORK/cache" "$WORK/data" "$WORK/state" "$WORK/temp"
export HOME="$WORK/home" XDG_CONFIG_HOME="$WORK/config" XDG_CACHE_HOME="$WORK/cache"
export XDG_DATA_HOME="$WORK/data" XDG_STATE_HOME="$WORK/state" TMPDIR="$WORK/temp"
python3 -I - <<'PYFIXTURE'
import os, pathlib, zipfile
with zipfile.ZipFile(pathlib.Path(os.environ['WORK']) / ':)', 'w') as archive:
    archive.writestr(':)', 'hi')
PYFIXTURE
node scripts/rea.mjs --version
node scripts/rea.mjs inspect-artifact "$WORK/:)" --format md > "$WORK/report.md"
node scripts/rea.mjs inspect-artifact "$WORK/:)" --format json > "$WORK/report.json"
node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { createMarkdownRenderer, disposeMdItInstance } from 'vitepress';
import { parseFragment } from 'parse5';
const root = process.env.WORK;
const source = readFileSync(`${root}/report.md`, 'utf8');
const data = JSON.parse(readFileSync(`${root}/report.json`, 'utf8'));
const text = n => n.nodeName === '#text' ? n.value : (n.childNodes ?? []).map(text).join('');
assert.equal(data.subject.name, ':)');
assert(source.includes('## subject.name\n\n:)'));
for (const disableEmoji of [false, true]) {
  const options = { linkify: false, ...(disableEmoji ? { emoji: { defs: {}, shortcuts: {} } } : {}) };
  const renderer = await createMarkdownRenderer(root, options);
  const nodes = parseFragment(renderer.render(source)).childNodes;
  const i = nodes.findIndex(n => n.tagName === 'h2' && text(n).replaceAll(String.fromCodePoint(0x200b), '').trim() === 'subject.name');
  assert(i >= 0);
  const body = text(nodes.slice(i + 1).find(n => n.tagName === 'p'));
  const cells = [];
  function visit(n) {
    if (n.tagName === 'td') cells.push(text(n));
    else (n.childNodes ?? []).forEach(visit);
  }
  nodes.forEach(visit);
  const marker = '"logical_path":"';
  const inventory = cells.find(cell => cell.includes(marker));
  assert(inventory);
  const paths = [...inventory.matchAll(/"logical_path":"([^"\n]*)"/g)].map(m => m[1]);
  const expected = disableEmoji ? ':)' : '😃';
  assert.equal(body, expected);
  assert(paths.includes(expected));
  console.log(JSON.stringify({ disableEmoji, jsonName: data.subject.name, renderedName: body, renderedEntryPaths: paths }));
  disposeMdItInstance();
}
JS

Read the comparison output. With default emoji support, the observed rendered name and entry path are 😃; the emoji-disabled control retains :). All CLI commands exit 0 with empty stderr. The assertion script exits 0 after confirming those results.

Expected behavior

The rendered name and entry path should remain :) when CLI output is rendered with the same locked VitePress configuration that the project's existing literal-display boundary tests use. Formatting should identify the original artifact and archive entry.

The CLI documentation expressly promises literal source-text display in Markdown table cells, including nested JSON field names and values. Its description of escaping as a presentation change also says retained Evidence and JSON keep the original values; this does not reduce the display contract to raw Markdown validity. The scalar/paragraph expectation additionally follows the formatter's literal-data comment and existing rendered scalar tests. Exact locators appear below.

Impact and scope

P2: an artifact and entry named :) appear with a different identifier in the successful rendered report. This is a bounded display-fidelity failure with directly verified workarounds: read the raw Markdown, use --format json, or disable emoji substitution in the renderer.

The verified scope is the subject.name paragraph and an inventory JSON table cell rendered by locked VitePress 1.6.4 with { linkify: false } and default emoji support. Ordinary standalone :) text is also transformed by that renderer; REA does not cause an unusual emoji-parser behavior. The defect is that the produced literal-data representation permits the transformation under the renderer REA already uses to check fidelity.

No source artifact modification or stored Evidence loss was established. This report makes no claim about GFM, arbitrary renderer plugins, other emoticons, headers, browser visual layout, other platforms, or frequency in real artifacts.

REA version

Exact stdout from node scripts/rea.mjs --version was 6.8.0 followed by one newline; exit 0. Baseline source commit: 65f1c447344a423fa2785847dbcea1e9a9e2454a.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1, arm64; checkout-locked VitePress 1.6.4 and parse5 8.0.1. No GUI or external analysis engine was launched.

Target and artifact context

An inline-generated 104-byte ZIP with one regular entry. The artifact basename and entry name are both the two literal characters :); its entry contents are hi.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The independent CLI/renderer replay printed:

6.8.0
{"disableEmoji":false,"jsonName":":)","renderedName":"😃","renderedEntryPaths":[".","😃"]}
{"disableEmoji":true,"jsonName":":)","renderedName":":)","renderedEntryPaths":[".",":)"]}

These outputs distinguish the original data, default rendered display, and emoji-disabled control. The inline assertions also confirm that raw Markdown contains the literal section body :).

Expected-source locators at the baseline:

docs/cli.md:965-973: literal source text in Markdown table headers/cells; nested JSON retains field names and values; retained Evidence and JSON preserve original values.
tests/boundary/cli/markdownLiteralCells.test.ts:80-97: public CLI Markdown is rendered with createMarkdownRenderer(root, { linkify: false }); rendered cell texts must equal the original values.
tests/boundary/cli/markdownLiteralCells.test.ts:108-157: rendered scalar and heading literal-data checks.
src/cli/cliOutputFormatter.ts:69: heading and paragraph facts are literal data.

Failure/provenance locators inspected separately from the executed CLI comparison:

src/cli/cliOutputFormatter.ts:61-77: shared table/text escaping protects other syntax but leaves :) available to the renderer.
docs/.vitepress/config.ts: no Markdown or emoji override. The inspected docs and boundary tests do not state that emoji substitution is disabled.
node_modules/vitepress/dist/node/chunk-D3CUZ4fa.js:35715-35743: the locked renderer installs the emoji plugin by default. Both tested default renderings use that behavior; the control explicitly removes emoji definitions and shortcuts.

Analysis or suspected cause

Source-derived explanation: the formatter emits the value as ordinary inline text, including inside its nested JSON cell. VitePress then interprets :) as an emoji shortcut. The emoji-disabled control preserves both positions, identifying that renderer extension as the transformation boundary. The existing escaping contract and tests cover other inline syntax but omit this input.

A restoration check should compare rendered paragraph text and the inventory entry path to the JSON/source :) under the existing default renderer oracle. Passing JSON alone or merely emitting valid Markdown would not establish that restoration.

Additional context

Duplicate searches covered open and closed issues, issue comments, and open/closed/merged PRs for Markdown, literal display, emoji, emoticon, shortcode and substitution terms.

Related reports have different closure checks: #7325 concerns brace-delimited provider cells disappearing; #7440 concerns parser-error pipes; #7432 and merged #7506 concern table edge spaces; #7625 concerns paragraph trailing spaces; #7592 and #7593 concern surrogate-byte and NUL-rendering substitutions respectively. Closed #6755 repairs table pipes, and closed #5530 with merged #5814 repairs token windows. This reproduction uses no token window.

Merged #7073 established literal inline table handling and the VitePress comparison, while merged #6932 handles HTML/entity text. Neither claims an emoticon fix. No regression of those specific fixes is alleged. First affected release and last known good release are unknown; current main was not separately executed.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A successful rendered artifact report changes the displayed artifact and archive-entry identifiers for a legal filename under the repository's existing renderer oracle. The observed failure is bounded and does not establish stored data loss.
Workaround status: available
Workaround basis: Paired JSON and raw Markdown retain ':)' in the executed comparisons; the executed emoji-disabled renderer also preserves both displayed positions.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7633 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area CLI ### Observed behavior For a ZIP named `:)` with an entry also named `:)`, `node scripts/rea.mjs inspect-artifact "$WORK/:)" --format md` succeeds. Rendering its output with the checkout's locked VitePress renderer displays `😃` as the artifact name under `subject.name`. The nested inventory JSON table cell also displays `"logical_path":"😃"` for the entry. The emitted Markdown and paired JSON retain `:)`. Substitution first occurs in the renderer, whose emoji plugin is enabled by default. A second renderer instance with empty emoji definitions/shortcuts preserves `:)` in both positions. Two independently executed CLI/renderer comparisons reproduced the substitution. ### Reproduction 1. Use the already-built baseline checkout with its locked dependencies installed and Node 24.22.1 available as `node`. Run the following from its root. It generates the ZIP inline, isolates HOME/XDG paths, compares the rendered output, and deletes its temporary files on exit: ```sh export WORK="$(mktemp -d)" trap 'rm -rf "$WORK"' EXIT mkdir -p "$WORK/home" "$WORK/config" "$WORK/cache" "$WORK/data" "$WORK/state" "$WORK/temp" export HOME="$WORK/home" XDG_CONFIG_HOME="$WORK/config" XDG_CACHE_HOME="$WORK/cache" export XDG_DATA_HOME="$WORK/data" XDG_STATE_HOME="$WORK/state" TMPDIR="$WORK/temp" python3 -I - <<'PYFIXTURE' import os, pathlib, zipfile with zipfile.ZipFile(pathlib.Path(os.environ['WORK']) / ':)', 'w') as archive: archive.writestr(':)', 'hi') PYFIXTURE node scripts/rea.mjs --version node scripts/rea.mjs inspect-artifact "$WORK/:)" --format md > "$WORK/report.md" node scripts/rea.mjs inspect-artifact "$WORK/:)" --format json > "$WORK/report.json" node --input-type=module <<'JS' import assert from 'node:assert/strict'; import { readFileSync } from 'node:fs'; import { createMarkdownRenderer, disposeMdItInstance } from 'vitepress'; import { parseFragment } from 'parse5'; const root = process.env.WORK; const source = readFileSync(`${root}/report.md`, 'utf8'); const data = JSON.parse(readFileSync(`${root}/report.json`, 'utf8')); const text = n => n.nodeName === '#text' ? n.value : (n.childNodes ?? []).map(text).join(''); assert.equal(data.subject.name, ':)'); assert(source.includes('## subject.name\n\n:)')); for (const disableEmoji of [false, true]) { const options = { linkify: false, ...(disableEmoji ? { emoji: { defs: {}, shortcuts: {} } } : {}) }; const renderer = await createMarkdownRenderer(root, options); const nodes = parseFragment(renderer.render(source)).childNodes; const i = nodes.findIndex(n => n.tagName === 'h2' && text(n).replaceAll(String.fromCodePoint(0x200b), '').trim() === 'subject.name'); assert(i >= 0); const body = text(nodes.slice(i + 1).find(n => n.tagName === 'p')); const cells = []; function visit(n) { if (n.tagName === 'td') cells.push(text(n)); else (n.childNodes ?? []).forEach(visit); } nodes.forEach(visit); const marker = '"logical_path":"'; const inventory = cells.find(cell => cell.includes(marker)); assert(inventory); const paths = [...inventory.matchAll(/"logical_path":"([^"\n]*)"/g)].map(m => m[1]); const expected = disableEmoji ? ':)' : '😃'; assert.equal(body, expected); assert(paths.includes(expected)); console.log(JSON.stringify({ disableEmoji, jsonName: data.subject.name, renderedName: body, renderedEntryPaths: paths })); disposeMdItInstance(); } JS ``` 2. Read the comparison output. With default emoji support, the observed rendered name and entry path are `😃`; the emoji-disabled control retains `:)`. All CLI commands exit 0 with empty stderr. The assertion script exits 0 after confirming those results. ### Expected behavior The rendered name and entry path should remain `:)` when CLI output is rendered with the same locked VitePress configuration that the project's existing literal-display boundary tests use. Formatting should identify the original artifact and archive entry. The CLI documentation expressly promises literal source-text display in Markdown table cells, including nested JSON field names and values. Its description of escaping as a presentation change also says retained Evidence and JSON keep the original values; this does not reduce the display contract to raw Markdown validity. The scalar/paragraph expectation additionally follows the formatter's literal-data comment and existing rendered scalar tests. Exact locators appear below. ### Impact and scope P2: an artifact and entry named `:)` appear with a different identifier in the successful rendered report. This is a bounded display-fidelity failure with directly verified workarounds: read the raw Markdown, use `--format json`, or disable emoji substitution in the renderer. The verified scope is the `subject.name` paragraph and an inventory JSON table cell rendered by locked VitePress 1.6.4 with `{ linkify: false }` and default emoji support. Ordinary standalone `:)` text is also transformed by that renderer; REA does not cause an unusual emoji-parser behavior. The defect is that the produced literal-data representation permits the transformation under the renderer REA already uses to check fidelity. No source artifact modification or stored Evidence loss was established. This report makes no claim about GFM, arbitrary renderer plugins, other emoticons, headers, browser visual layout, other platforms, or frequency in real artifacts. ### REA version Exact stdout from `node scripts/rea.mjs --version` was `6.8.0` followed by one newline; exit 0. Baseline source commit: `65f1c447344a423fa2785847dbcea1e9a9e2454a`. ### Runtime and platform Node.js v24.22.1; macOS 27.0.1, arm64; checkout-locked VitePress 1.6.4 and parse5 8.0.1. No GUI or external analysis engine was launched. ### Target and artifact context An inline-generated 104-byte ZIP with one regular entry. The artifact basename and entry name are both the two literal characters `:)`; its entry contents are `hi`. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The independent CLI/renderer replay printed: ```text 6.8.0 {"disableEmoji":false,"jsonName":":)","renderedName":"😃","renderedEntryPaths":[".","😃"]} {"disableEmoji":true,"jsonName":":)","renderedName":":)","renderedEntryPaths":[".",":)"]} ``` These outputs distinguish the original data, default rendered display, and emoji-disabled control. The inline assertions also confirm that raw Markdown contains the literal section body `:)`. Expected-source locators at the baseline: - `docs/cli.md:965-973`: literal source text in Markdown table headers/cells; nested JSON retains field names and values; retained Evidence and JSON preserve original values. - `tests/boundary/cli/markdownLiteralCells.test.ts:80-97`: public CLI Markdown is rendered with `createMarkdownRenderer(root, { linkify: false })`; rendered cell texts must equal the original values. - `tests/boundary/cli/markdownLiteralCells.test.ts:108-157`: rendered scalar and heading literal-data checks. - `src/cli/cliOutputFormatter.ts:69`: heading and paragraph facts are literal data. Failure/provenance locators inspected separately from the executed CLI comparison: - `src/cli/cliOutputFormatter.ts:61-77`: shared table/text escaping protects other syntax but leaves `:)` available to the renderer. - `docs/.vitepress/config.ts`: no Markdown or emoji override. The inspected docs and boundary tests do not state that emoji substitution is disabled. - `node_modules/vitepress/dist/node/chunk-D3CUZ4fa.js:35715-35743`: the locked renderer installs the emoji plugin by default. Both tested default renderings use that behavior; the control explicitly removes emoji definitions and shortcuts. ### Analysis or suspected cause Source-derived explanation: the formatter emits the value as ordinary inline text, including inside its nested JSON cell. VitePress then interprets `:)` as an emoji shortcut. The emoji-disabled control preserves both positions, identifying that renderer extension as the transformation boundary. The existing escaping contract and tests cover other inline syntax but omit this input. A restoration check should compare rendered paragraph text and the inventory entry path to the JSON/source `:)` under the existing default renderer oracle. Passing JSON alone or merely emitting valid Markdown would not establish that restoration. ### Additional context Duplicate searches covered open and closed issues, issue comments, and open/closed/merged PRs for Markdown, literal display, emoji, emoticon, shortcode and substitution terms. Related reports have different closure checks: [#7325](https://github.com/morluto/rea/issues/7325) concerns brace-delimited provider cells disappearing; [#7440](https://github.com/morluto/rea/issues/7440) concerns parser-error pipes; [#7432](https://github.com/morluto/rea/issues/7432) and merged [#7506](https://github.com/morluto/rea/pull/7506) concern table edge spaces; [#7625](https://github.com/morluto/rea/issues/7625) concerns paragraph trailing spaces; [#7592](https://github.com/morluto/rea/issues/7592) and [#7593](https://github.com/morluto/rea/issues/7593) concern surrogate-byte and NUL-rendering substitutions respectively. Closed [#6755](https://github.com/morluto/rea/issues/6755) repairs table pipes, and closed [#5530](https://github.com/morluto/rea/issues/5530) with merged [#5814](https://github.com/morluto/rea/pull/5814) repairs token windows. This reproduction uses no token window. Merged [#7073](https://github.com/morluto/rea/pull/7073) established literal inline table handling and the VitePress comparison, while merged [#6932](https://github.com/morluto/rea/pull/6932) handles HTML/entity text. Neither claims an emoticon fix. No regression of those specific fixes is alleged. First affected release and last known good release are unknown; current main was not separately executed. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A successful rendered artifact report changes the displayed artifact and archive-entry identifiers for a legal filename under the repository's existing renderer oracle. The observed failure is bounded and does not establish stored data loss. - Workaround status: available - Workaround basis: Paired JSON and raw Markdown retain ':)' in the executed comparisons; the executed emoji-disabled renderer also preserves both displayed positions.

bsakaguchi referenced this issue

2026-10-31 01:13:28 +00:00

[PR #4563] [CLOSED] test(firmware): assert composed publication cleanup failures #6309

bsakaguchi referenced this issue

2026-11-01 21:10:53 +00:00

[GH-ISSUE #3582] [Tracker] Ready to close or merge: issues and pull requests (October 2026) #1330

bsakaguchi closed this issue

2026-11-01 21:11:11 +00:00

bsakaguchi commented

2026-11-01 21:11:12 +00:00

Author

Owner

@coygeek commented on GitHub (Oct 31, 2026):

I checked this against latest main. This appears resolved. The problem was that the CLI Markdown formatter emitted a literal :) artifact/entry name as ordinary inline text, so the locked VitePress renderer's default emoji plugin substituted 😃 for it in the rendered report — both in the paragraph under subject.name and in the nested inventory JSON table cell showing "logical_path":"😃", even though the raw Markdown and JSON correctly retained :). PR #7938 (merge commit e124bdfc5, merged into main) fixes this at the shared formatter boundary in src/cli/cliOutputFormatter.ts: escapeTableCell now encodes : as :, so the emitted Markdown no longer presents the literal :) emoji-shortcut sequence to the renderer, while the rendered text still reads as :). Because paragraph facts go through escapeMarkdownText, which delegates to escapeTableCell, the same encoding covers both the subject.name paragraph and the inventory JSON table cell. The PR's regression tests confirm this under the locked VitePress renderer with { linkify: false }, asserting a scalar ":)" renders as literal paragraph text and that rendered table cells contain ":)" and the nested JSON. I verified the merge commit is an ancestor of main, so this fix is present on the default branch. Closing as resolved.

@coygeek commented on GitHub (Oct 31, 2026): I checked this against latest main. This appears resolved. The problem was that the CLI Markdown formatter emitted a literal `:) ` artifact/entry name as ordinary inline text, so the locked VitePress renderer's default emoji plugin substituted `😃` for it in the rendered report — both in the paragraph under `subject.name` and in the nested inventory JSON table cell showing `"logical_path":"😃"`, even though the raw Markdown and JSON correctly retained `:)`. [PR #7938](https://github.com/morluto/rea/pull/7938) (merge commit [`e124bdfc5`](https://github.com/morluto/rea/commit/e124bdfc54707da138a58ae61b901f0bf93a9bac), merged into main) fixes this at the shared formatter boundary in `src/cli/cliOutputFormatter.ts`: `escapeTableCell` now encodes `:` as `:`, so the emitted Markdown no longer presents the literal `:)` emoji-shortcut sequence to the renderer, while the rendered text still reads as `:)`. Because paragraph facts go through `escapeMarkdownText`, which delegates to `escapeTableCell`, the same encoding covers both the `subject.name` paragraph and the inventory JSON table cell. The PR's regression tests confirm this under the locked VitePress renderer with `{ linkify: false }`, asserting a scalar `":)"` renders as literal paragraph text and that rendered table cells contain `":)"` and the nested JSON. I verified the merge commit is an ancestor of main, so this fix is present on the default branch. Closing as resolved.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#4558

No description provided.

Rows
Columns