morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #4612] [Bug] Resource locale discovery silently omits locale tables with additional qualifiers #2396

New issue

Closed

opened 2026-10-17 21:04:07 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-17 21:04:07 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 16, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/4612

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Native analysis or decompilation

Observed behavior

The public decode-android-resources command omits locale-bearing resource directories that also specify another Android configuration qualifier. On both tested builds, a controlled launcher writes strings tables under values-en-rUS-v21, values-b+zh+Hans+CN-night and values-pt-rBR-sw600dp, but the returned locales contains only the b+sr+Latn and fr controls. The result reports coverage: "complete", and its limitations do not disclose those omissions. Default strings and explicit French and plain BCP 47 string selection still work.

This was observed through the real REA CLI with a simulated decoded workspace. The fixture launcher supplies decoded files directly and reports a fixture version string; it does not run Apktool or decode a real APK.

Reproduction

Save the following as reproduce.py in a writable scratch directory. It requires Python 3, a POSIX shell and a supported Node.js on PATH. The launcher supplies decoded files directly; the input is a placeholder, not a real APK. Every REA invocation receives a new HOME and TMPDIR and an allowlisted environment.

from pathlib import Path
import json, os, shutil, subprocess, sys, tempfile

cli = Path(sys.argv[1]).resolve()
work = Path(tempfile.mkdtemp(prefix="rea-locale-", dir=Path.cwd()))
node = shutil.which("node")
assert node is not None
launcher = work / "fixture-apktool"
launcher.write_text(r'''#!/bin/sh
set -eu
if [ "$1" = --version ]; then printf '2.7.0\n'; exit 0; fi
[ "$1" = d ] || exit 2
out=
while [ "$#" -gt 0 ]; do
  if [ "$1" = -o ]; then shift; out="$1"; break; fi
  shift
done
[ -n "$out" ] || exit 2
printf 'versionInfo:\n  versionName: 1.0\n  versionCode: 1\nsdkInfo:\n  minSdkVersion: 21\n' > "$out/apktool.yml"
printf '<manifest package="org.example.qualifiers"/>' > "$out/AndroidManifest.xml"
for q in values values-fr values-b+sr+Latn values-b+zh+Hans+CN-night values-en-rUS-v21 values-pt-rBR-sw600dp values-night; do
  mkdir -p "$out/res/$q"
  printf '<resources><string name="marker">%s</string></resources>' "$q" > "$out/res/$q/strings.xml"
done
''')
launcher.chmod(0o755)
apk = work / "input.apk"
apk.write_bytes(b"fixture-input")
try:
    for locale in (None, None, "fr", "b+sr+Latn", "en-rUS"):
        home = tempfile.mkdtemp(prefix="home-", dir=work)
        temporary = tempfile.mkdtemp(prefix="runtime-", dir=work)
        env = {"HOME": home, "TMPDIR": temporary, "PATH": str(Path(node).parent) + os.pathsep + os.defpath, "REA_APKTOOL_COMMAND": str(launcher), "NO_COLOR": "1"}
        args = [node, str(cli), "decode-android-resources", str(apk), "--json"]
        if locale is not None:
            args += ["--locale", locale]
        run = subprocess.run(args, env=env, capture_output=True, text=True, timeout=40)
        assert run.returncode == 0, (run.returncode, run.stdout, run.stderr)
        result = json.loads(run.stdout)
        facts = result["normalized_result"]
        print(json.dumps({"requested_locale": locale, "exit": run.returncode, "locales": facts["locales"], "strings": facts["strings"], "coverage": facts["coverage"], "decoded_file_count": facts["decoded_file_count"], "limitations": result["limitations"]}))
finally:
    shutil.rmtree(work)

For an installed package, run python3 reproduce.py /path/to/node_modules/rea-agents/scripts/rea.mjs. For a fresh source checkout, run npm ci and npm run build:cached, then pass /path/to/rea/scripts/rea.mjs instead. Both paths were exercised independently.
Observe that both default calls omit all three combined configurations without a corresponding limitation. The French and plain BCP 47 controls are discoverable and selectable. Across both builds, all four default calls reproduced the omission; each explicit control ran once per build with the results below.

Expected behavior

Locale discovery should account for locale-bearing string tables even when another Android configuration qualifier appears in the directory name, or explicitly disclose that discovery is incomplete. REA advertises available string locales and documents discovery of locale strings tables. Android permits multiple dash-separated resource qualifiers in their prescribed order and documents BCP 47 locale tags, night mode and platform-version qualifiers in its resource configuration guide. A locale that exists only in a combined configuration should not silently disappear from a result presented as complete.

Impact and scope

The observed result hides three locale-bearing configurations in a small decoded-resource fixture on every default run tested across both builds. An analyst relying on the inventory cannot discover those tables from this response. This is a bounded resource-discovery defect; default strings, manifest projection and locale-only tables continue to work. Frequency across real APKs and behavior of an actual Apktool decode were not measured.

Impact level: P2
Assessment status: supported
Impact basis: The CLI returns an incomplete locale inventory with complete coverage and no omission notice for valid Android resource configurations; the demonstrated failure affects one part of resource analysis rather than blocking all decoding.
Workaround status: unverified
Workaround basis: Decoding and inspecting resource directories outside REA may recover the inventory, but no real external-decoder workaround was tested. Requesting the known locale en-rUS returned no strings for its version-qualified table.

REA version

Published npm package rea-agents@6.5.0 (latest npm and GitHub release checked); fresh source build of current main at d5be402fb3dbfc56ac9854c27b5538df20cc2437. Both CLI version commands returned 6.5.0.

Runtime and platform

Node.js v26.12.0; macOS 27.0.1, Darwin arm64. The launcher is a shell fixture that reports 2.7.0, not an installed Apktool engine.

Target and artifact context

Synthetic decoded Android resources with a text manifest, apktool.yml and seven strings tables. The public CLI snapshots a 13-byte placeholder input file; the configured launcher then emits the resource fixture. No production artifact or real APK decode was involved.

Interface

CLI

Earliest failure stage

Capture or observation

Relevant output or evidence

Expected source: src/contracts/apktool/apktoolToolContracts.ts:25 advertises available string locales; docs/apktool-resource-analysis.md:64 describes locale strings-table discovery.
Failure source: src/apktool/ApktoolProvider.ts:314 validates the entire resource-directory suffix as a locale before adding it to the inventory.
Evidence provenance: observed
Local verification: reproduced
Reproduction completeness: complete
Supporting source: src/domain/apktool/apktoolResourceAnalysis.ts:17 defines the locale-only grammar; src/apktool/ApktoolProvider.ts:773 derives coverage without tracking this filtering, and src/apktool/ApktoolProvider.ts:805 builds the limitations without a corresponding omission notice.

Both independently installed/built CLIs returned exit 0 on their default calls with these selected fields:

{
  "locales": ["b+sr+Latn", "fr"],
  "strings": [{"name": "marker", "value": "values"}],
  "decoded_file_count": 9,
  "coverage": "complete",
  "limitations": [
    "Artifact identity is unavailable for this observation.",
    "Decoding skips smali (--no-src); Java-source recovery belongs to the JADX family"
  ]
}

The control results were:

fr: exit 0; strings = [{"name":"marker","value":"values-fr"}]
b+sr+Latn: exit 0; strings = [{"name":"marker","value":"values-b+sr+Latn"}]
en-rUS: exit 0; strings = []; limitation = "No strings.xml was found for locale en-rUS"

Typed outcome or recovery expected

For the same fixture, the current failure is an inventory containing only the two locale-only controls with complete coverage and no notice about the combined configurations. A passing result must account for the locale-bearing combined configurations or explicitly identify the incomplete discovery. Retain discovery and string projection for fr and b+sr+Latn, preserve default string projection, and continue excluding configuration-only values-night from the locale list. This does not require treating a complete resource-configuration suffix as a selectable locale tag.

Analysis or suspected cause

Source-derived reasoning from the main checkout: discovery applies the request's locale-only validator to the whole resource-configuration suffix. A suffix containing a locale and an additional configuration qualifier fails that validator, so discovery discards it without recording an omission. The failure occurs before CLI result serialization.

Additional context

The existing test at src/apktool/ApktoolProvider.test.ts:266 checks exclusion of configuration-only car, night and v21; it does not cover a locale combined with another qualifier. No known-good release was established.

Related closed issue morluto/rea#2563 addresses configuration-only directories being advertised as locales. This report concerns locale-bearing directories with additional qualifiers being omitted. The fixture keeps a configuration-only night table as a discriminating control.

Originally created by @coygeek on GitHub (Oct 16, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/4612 ### Before submitting - [X] I searched the open and closed issues for a related report. - [X] I removed credentials, tokens, private paths, and sensitive target data. ### Area Native analysis or decompilation ### Observed behavior The public `decode-android-resources` command omits locale-bearing resource directories that also specify another Android configuration qualifier. On both tested builds, a controlled launcher writes strings tables under `values-en-rUS-v21`, `values-b+zh+Hans+CN-night` and `values-pt-rBR-sw600dp`, but the returned `locales` contains only the `b+sr+Latn` and `fr` controls. The result reports `coverage: "complete"`, and its limitations do not disclose those omissions. Default strings and explicit French and plain BCP 47 string selection still work. This was observed through the real REA CLI with a simulated decoded workspace. The fixture launcher supplies decoded files directly and reports a fixture version string; it does not run Apktool or decode a real APK. ### Reproduction 1. Save the following as `reproduce.py` in a writable scratch directory. It requires Python 3, a POSIX shell and a supported Node.js on PATH. The launcher supplies decoded files directly; the input is a placeholder, not a real APK. Every REA invocation receives a new HOME and TMPDIR and an allowlisted environment. ```python from pathlib import Path import json, os, shutil, subprocess, sys, tempfile cli = Path(sys.argv[1]).resolve() work = Path(tempfile.mkdtemp(prefix="rea-locale-", dir=Path.cwd())) node = shutil.which("node") assert node is not None launcher = work / "fixture-apktool" launcher.write_text(r'''#!/bin/sh set -eu if [ "$1" = --version ]; then printf '2.7.0\n'; exit 0; fi [ "$1" = d ] || exit 2 out= while [ "$#" -gt 0 ]; do if [ "$1" = -o ]; then shift; out="$1"; break; fi shift done [ -n "$out" ] || exit 2 printf 'versionInfo:\n versionName: 1.0\n versionCode: 1\nsdkInfo:\n minSdkVersion: 21\n' > "$out/apktool.yml" printf '<manifest package="org.example.qualifiers"/>' > "$out/AndroidManifest.xml" for q in values values-fr values-b+sr+Latn values-b+zh+Hans+CN-night values-en-rUS-v21 values-pt-rBR-sw600dp values-night; do mkdir -p "$out/res/$q" printf '<resources><string name="marker">%s</string></resources>' "$q" > "$out/res/$q/strings.xml" done ''') launcher.chmod(0o755) apk = work / "input.apk" apk.write_bytes(b"fixture-input") try: for locale in (None, None, "fr", "b+sr+Latn", "en-rUS"): home = tempfile.mkdtemp(prefix="home-", dir=work) temporary = tempfile.mkdtemp(prefix="runtime-", dir=work) env = {"HOME": home, "TMPDIR": temporary, "PATH": str(Path(node).parent) + os.pathsep + os.defpath, "REA_APKTOOL_COMMAND": str(launcher), "NO_COLOR": "1"} args = [node, str(cli), "decode-android-resources", str(apk), "--json"] if locale is not None: args += ["--locale", locale] run = subprocess.run(args, env=env, capture_output=True, text=True, timeout=40) assert run.returncode == 0, (run.returncode, run.stdout, run.stderr) result = json.loads(run.stdout) facts = result["normalized_result"] print(json.dumps({"requested_locale": locale, "exit": run.returncode, "locales": facts["locales"], "strings": facts["strings"], "coverage": facts["coverage"], "decoded_file_count": facts["decoded_file_count"], "limitations": result["limitations"]})) finally: shutil.rmtree(work) ``` 2. For an installed package, run `python3 reproduce.py /path/to/node_modules/rea-agents/scripts/rea.mjs`. For a fresh source checkout, run `npm ci` and `npm run build:cached`, then pass `/path/to/rea/scripts/rea.mjs` instead. Both paths were exercised independently. 3. Observe that both default calls omit all three combined configurations without a corresponding limitation. The French and plain BCP 47 controls are discoverable and selectable. Across both builds, all four default calls reproduced the omission; each explicit control ran once per build with the results below. ### Expected behavior Locale discovery should account for locale-bearing string tables even when another Android configuration qualifier appears in the directory name, or explicitly disclose that discovery is incomplete. REA advertises available string locales and documents discovery of locale strings tables. Android permits multiple dash-separated resource qualifiers in their prescribed order and documents BCP 47 locale tags, night mode and platform-version qualifiers in its [resource configuration guide](https://developer.android.com/guide/topics/resources/providing-resources#AlternativeResources). A locale that exists only in a combined configuration should not silently disappear from a result presented as complete. ### Impact and scope The observed result hides three locale-bearing configurations in a small decoded-resource fixture on every default run tested across both builds. An analyst relying on the inventory cannot discover those tables from this response. This is a bounded resource-discovery defect; default strings, manifest projection and locale-only tables continue to work. Frequency across real APKs and behavior of an actual Apktool decode were not measured. - Impact level: P2 - Assessment status: supported - Impact basis: The CLI returns an incomplete locale inventory with complete coverage and no omission notice for valid Android resource configurations; the demonstrated failure affects one part of resource analysis rather than blocking all decoding. - Workaround status: unverified - Workaround basis: Decoding and inspecting resource directories outside REA may recover the inventory, but no real external-decoder workaround was tested. Requesting the known locale en-rUS returned no strings for its version-qualified table. ### REA version Published npm package `rea-agents@6.5.0` (latest npm and GitHub release checked); fresh source build of current main at `d5be402fb3dbfc56ac9854c27b5538df20cc2437`. Both CLI version commands returned `6.5.0`. ### Runtime and platform Node.js v26.12.0; macOS 27.0.1, Darwin arm64. The launcher is a shell fixture that reports 2.7.0, not an installed Apktool engine. ### Target and artifact context Synthetic decoded Android resources with a text manifest, apktool.yml and seven strings tables. The public CLI snapshots a 13-byte placeholder input file; the configured launcher then emits the resource fixture. No production artifact or real APK decode was involved. ### Interface CLI ### Earliest failure stage Capture or observation ### Relevant output or evidence - Expected source: src/contracts/apktool/apktoolToolContracts.ts:25 advertises available string locales; docs/apktool-resource-analysis.md:64 describes locale strings-table discovery. - Failure source: src/apktool/ApktoolProvider.ts:314 validates the entire resource-directory suffix as a locale before adding it to the inventory. - Evidence provenance: observed - Local verification: reproduced - Reproduction completeness: complete - Supporting source: src/domain/apktool/apktoolResourceAnalysis.ts:17 defines the locale-only grammar; src/apktool/ApktoolProvider.ts:773 derives coverage without tracking this filtering, and src/apktool/ApktoolProvider.ts:805 builds the limitations without a corresponding omission notice. Both independently installed/built CLIs returned exit 0 on their default calls with these selected fields: ```json { "locales": ["b+sr+Latn", "fr"], "strings": [{"name": "marker", "value": "values"}], "decoded_file_count": 9, "coverage": "complete", "limitations": [ "Artifact identity is unavailable for this observation.", "Decoding skips smali (--no-src); Java-source recovery belongs to the JADX family" ] } ``` The control results were: ```text fr: exit 0; strings = [{"name":"marker","value":"values-fr"}] b+sr+Latn: exit 0; strings = [{"name":"marker","value":"values-b+sr+Latn"}] en-rUS: exit 0; strings = []; limitation = "No strings.xml was found for locale en-rUS" ``` ### Typed outcome or recovery expected For the same fixture, the current failure is an inventory containing only the two locale-only controls with complete coverage and no notice about the combined configurations. A passing result must account for the locale-bearing combined configurations or explicitly identify the incomplete discovery. Retain discovery and string projection for `fr` and `b+sr+Latn`, preserve default string projection, and continue excluding configuration-only `values-night` from the locale list. This does not require treating a complete resource-configuration suffix as a selectable locale tag. ### Analysis or suspected cause Source-derived reasoning from the main checkout: discovery applies the request's locale-only validator to the whole resource-configuration suffix. A suffix containing a locale and an additional configuration qualifier fails that validator, so discovery discards it without recording an omission. The failure occurs before CLI result serialization. ### Additional context The existing test at `src/apktool/ApktoolProvider.test.ts:266` checks exclusion of configuration-only `car`, `night` and `v21`; it does not cover a locale combined with another qualifier. No known-good release was established. Related closed issue [morluto/rea#2563](https://github.com/morluto/rea/issues/2563) addresses configuration-only directories being advertised as locales. This report concerns locale-bearing directories with additional qualifiers being omitted. The fixture keeps a configuration-only night table as a discriminating control.

bsakaguchi closed this issue

2026-10-17 21:04:08 +00:00

bsakaguchi commented

2026-10-17 21:04:09 +00:00

Author

Owner

@morluto commented on GitHub (Oct 17, 2026):

Fixed in 98886addd (pushed to main), adapting the disclosure policy from #4738 with contributor credit.

The inventory now consumes observed entry kinds from the owned traversal. Exact selectable locale names share request-alias normalization; other resource configurations retain their raw names in limitations. Non-regular table paths are disclosed with partial coverage, and inspection failures remain failures. The fixture separator was also corrected so all claimed tables actually exist.

Validation: 164 tests across 15 files, check:fast, test discovery, docs:check, commit/push hooks; two regression controls fail on the baseline; two source reviews found no blockers. Twenty-four tiny authored CLI/MCP comparisons preserved ordinary Evidence and verified qualified/unknown/non-regular disclosures. Warm CLI elapsed baseline/candidate: plain 0.889/0.906s, qualified 0.891/0.882s; CPU 1.112/1.121s and 1.128/1.105s; sampled family RSS about 517/516 MB and 516/515 MB. MCP full-discovery workflows: plain 2.802/2.809s, qualified 2.790/2.748s; CPU 3.920/3.917s and 3.927/3.686s; sampled family RSS 1352/1357 MB and 1354/1269 MB. cl100k output estimates: CLI 498 unchanged / 513→648; MCP including full discovery 804429 unchanged / 804459→804756.

These small authored-launcher measurements establish no speedup, real Apktool/JVM behavior, native Windows coverage, true cold costs, or large-inventory serialization performance. Request grammar is unchanged; longer unsupported BCP47 suffixes are disclosed rather than advertised as selectable. Android runtime resource matching is not inferred.

@morluto commented on GitHub (Oct 17, 2026): Fixed in 98886addd (pushed to main), adapting the disclosure policy from #4738 with contributor credit. The inventory now consumes observed entry kinds from the owned traversal. Exact selectable locale names share request-alias normalization; other resource configurations retain their raw names in limitations. Non-regular table paths are disclosed with partial coverage, and inspection failures remain failures. The fixture separator was also corrected so all claimed tables actually exist. Validation: 164 tests across 15 files, check:fast, test discovery, docs:check, commit/push hooks; two regression controls fail on the baseline; two source reviews found no blockers. Twenty-four tiny authored CLI/MCP comparisons preserved ordinary Evidence and verified qualified/unknown/non-regular disclosures. Warm CLI elapsed baseline/candidate: plain 0.889/0.906s, qualified 0.891/0.882s; CPU 1.112/1.121s and 1.128/1.105s; sampled family RSS about 517/516 MB and 516/515 MB. MCP full-discovery workflows: plain 2.802/2.809s, qualified 2.790/2.748s; CPU 3.920/3.917s and 3.927/3.686s; sampled family RSS 1352/1357 MB and 1354/1269 MB. cl100k output estimates: CLI 498 unchanged / 513→648; MCP including full discovery 804429 unchanged / 804459→804756. These small authored-launcher measurements establish no speedup, real Apktool/JVM behavior, native Windows coverage, true cold costs, or large-inventory serialization performance. Request grammar is unchanged; longer unsupported BCP47 suffixes are disclosed rather than advertised as selectable. Android runtime resource matching is not inferred.

bsakaguchi referenced this issue

2026-10-31 00:29:41 +00:00

[GH-ISSUE #2383] [Tracker] Audit analysis and debugging coverage for the TIOBE top 100 languages (October 2026) #794

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#2396

No description provided.

Rows
Columns