[GH-ISSUE #4694] [Bug] Apktool drops the entire string table when valid XML contains U+FFFD #2449

New issue

Closed

opened 2026-10-17 21:04:49 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-17 21:04:49 +00:00

Owner

Originally created by @ys0v9 on GitHub (Oct 17, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4694

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

decode-android-resources loses every string in the default table when one valid UTF-8 string contains a literal U+FFFD (�). Real Apktool successfully decodes both entries, but REA returns strings: [], coverage: "partial", and a malformed-XML limitation.

Reproduction

After npm ci and npm run build:cached, run this reduced reproduction from the repository root:

node --input-type=module <<'JS'
import { parseDecodedStrings } from './dist/apktool/ApktoolDecodeOutput.js';
for (const value of ['plain', '\uFFFD']) {
  const xml = `<resources><string name="good">Hello</string><string name="edge">${value}</string></resources>`;
  console.log(JSON.stringify({ value, ...parseDecodedStrings(xml) }));
}
JS

The public CLI was also exercised using two synthetic, unsigned APKs built with the AAPT2 binary and Android framework bundled in official Apktool 3.0.3. Both use package org.rea.unicodeprobe, a minimal manifest, and the two string entries above. Only edge differs. No APK was installed or executed.

REA_APKTOOL_COMMAND=/path/to/apktool node scripts/rea.mjs decode-android-resources /path/to/fixture.apk --json

The control and U+FFFD cases each reproduced consistently in two independent CLI runs. Separately decoding each APK with apktool d --no-src retained both expected strings.

Expected behavior

Return good=Hello and edge=�, with complete string coverage for this fixture. The resource-analysis guide promises verbatim projection of string values. U+FFFD is allowed by XML 1.0's Char production; its literal presence is not proof of invalid UTF-8 or malformed XML.

Impact and scope

Observed: one affected entry suppresses its unaffected sibling as well. This is a static resource-analysis result defect, reproduced 2/2 times per fixture. Other OSes, other Apktool versions, and MCP were not tested. The shared XML parser also serves manifest package extraction; manifest impact is source-derived here.

REA version

6.5.0, source revision be0b2d138c46bb951fc3c8ee57ef7b0d57fed312. The affected parser is unchanged at 5717ad24aa6471aa4207c370e26274866fb8aee5.

Runtime and platform

Node.js 24.18.0; npm 11.16.0; macOS 26.5.1 arm64; Apktool 3.0.3; OpenJDK 21.0.10.

Target and artifact context

Synthetic Android APK containing a compiled resource table. Failing APK SHA-256: 7a43b05965c43e05360cc47bfbac3dc14d606428e7ab34def0188a3a338453c2.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

Reduced reproduction:

{"value":"plain","entries":[{"name":"good","value":"Hello"},{"name":"edge","value":"plain"}],"unparsedCount":0}
{"value":"�","entries":[],"unparsedCount":1}

Real-provider CLI results (exit code 0 in all four runs):

Fixture	Apktool decoded entries	REA strings	REA coverage
`edge=plain`	`edge=plain`, `good=Hello`	Both entries	`complete`
`edge=�`	`edge=�`, `good=Hello`	`[]`	`partial`

Additional limitation on the failing fixture:

String entries omitted due to malformed XML or unsupported markup: 1

Analysis or suspected cause

Source-derived: parseXml treats every xmldom diagnostic as an error. The pinned xmldom emits the warning Unicode replacement character detected, source encoding issues? for any literal U+FFFD. parseDecodedStrings catches this and discards the entire document. The provider already uses a fatal UTF-8 decoder before XML parsing.

Additional context

Open and closed issue/PR searches for ApktoolDecodeOutput, parseDecodedStrings, apktool FFFD, and Apktool replacement-character reports found no matching report as of 2026-10-17. Related #2863 concerns SAZ session metadata, not the Apktool parser. #4612 concerns locale-directory discovery with extra qualifiers; this reproduction uses the default res/values/strings.xml without locale selection. Last known good version: NOT_ENOUGH_INFO; no version bisection performed.

Originally created by @ys0v9 on GitHub (Oct 17, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4694 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior `decode-android-resources` loses every string in the default table when one valid UTF-8 string contains a literal U+FFFD (`�`). Real Apktool successfully decodes both entries, but REA returns `strings: []`, `coverage: "partial"`, and a malformed-XML limitation. ### Reproduction After `npm ci` and `npm run build:cached`, run this reduced reproduction from the repository root: ```sh node --input-type=module <<'JS' import { parseDecodedStrings } from './dist/apktool/ApktoolDecodeOutput.js'; for (const value of ['plain', '\uFFFD']) { const xml = `<resources><string name="good">Hello</string><string name="edge">${value}</string></resources>`; console.log(JSON.stringify({ value, ...parseDecodedStrings(xml) })); } JS ``` The public CLI was also exercised using two synthetic, unsigned APKs built with the AAPT2 binary and Android framework bundled in official Apktool 3.0.3. Both use package `org.rea.unicodeprobe`, a minimal manifest, and the two string entries above. Only `edge` differs. No APK was installed or executed. ```sh REA_APKTOOL_COMMAND=/path/to/apktool node scripts/rea.mjs decode-android-resources /path/to/fixture.apk --json ``` The control and U+FFFD cases each reproduced consistently in two independent CLI runs. Separately decoding each APK with `apktool d --no-src` retained both expected strings. ### Expected behavior Return `good=Hello` and `edge=�`, with complete string coverage for this fixture. [The resource-analysis guide](https://github.com/morluto/rea/blob/main/docs/apktool-resource-analysis.md#interpret-the-result) promises verbatim projection of string values. U+FFFD is allowed by [XML 1.0's Char production](https://www.w3.org/TR/xml/#charsets); its literal presence is not proof of invalid UTF-8 or malformed XML. ### Impact and scope Observed: one affected entry suppresses its unaffected sibling as well. This is a static resource-analysis result defect, reproduced 2/2 times per fixture. Other OSes, other Apktool versions, and MCP were not tested. The shared XML parser also serves manifest package extraction; manifest impact is source-derived here. ### REA version 6.5.0, source revision `be0b2d138c46bb951fc3c8ee57ef7b0d57fed312`. The affected parser is unchanged at `5717ad24aa6471aa4207c370e26274866fb8aee5`. ### Runtime and platform Node.js 24.18.0; npm 11.16.0; macOS 26.5.1 arm64; Apktool 3.0.3; OpenJDK 21.0.10. ### Target and artifact context Synthetic Android APK containing a compiled resource table. Failing APK SHA-256: `7a43b05965c43e05360cc47bfbac3dc14d606428e7ab34def0188a3a338453c2`. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence Reduced reproduction: ```json {"value":"plain","entries":[{"name":"good","value":"Hello"},{"name":"edge","value":"plain"}],"unparsedCount":0} {"value":"�","entries":[],"unparsedCount":1} ``` Real-provider CLI results (exit code 0 in all four runs): | Fixture | Apktool decoded entries | REA strings | REA coverage | | --- | --- | --- | --- | | `edge=plain` | `edge=plain`, `good=Hello` | Both entries | `complete` | | `edge=�` | `edge=�`, `good=Hello` | `[]` | `partial` | Additional limitation on the failing fixture: ```text String entries omitted due to malformed XML or unsupported markup: 1 ``` ### Analysis or suspected cause Source-derived: [`parseXml`](https://github.com/morluto/rea/blob/5717ad24aa6471aa4207c370e26274866fb8aee5/src/apktool/ApktoolDecodeOutput.ts#L149-L159) treats every xmldom diagnostic as an error. The pinned xmldom emits the warning `Unicode replacement character detected, source encoding issues?` for any literal U+FFFD. `parseDecodedStrings` catches this and discards the entire document. The provider already uses a fatal UTF-8 decoder before XML parsing. ### Additional context Open and closed issue/PR searches for `ApktoolDecodeOutput`, `parseDecodedStrings`, `apktool FFFD`, and Apktool replacement-character reports found no matching report as of 2026-10-17. Related #2863 concerns SAZ session metadata, not the Apktool parser. #4612 concerns locale-directory discovery with extra qualifiers; this reproduction uses the default `res/values/strings.xml` without locale selection. Last known good version: NOT_ENOUGH_INFO; no version bisection performed.

bsakaguchi closed this issue

2026-10-17 21:04:50 +00:00

bsakaguchi referenced this issue

2026-10-31 01:05:03 +00:00

[PR #2449] [MERGED] fix(ida): retain bounded stderr on upstream transport failures #5238