morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #7545] [Bug] Harmony graph counts a known ZIP symlink as unavailable file content #4486

New issue

Closed

opened 2026-10-31 00:50:12 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:50:12 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 27, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/7545

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

After inspecting a generated HAP, node scripts/rea.mjs project-harmony-application-graph <input.json> --format json accepts the inspector's Evidence but reports the known ZIP symlink ets/current in unavailable_components. It also returns coverage.status: "partial" and a limitation saying component and runtime absence is unknown.

The input inventory reports complete container-entry coverage and identifies that occurrence as a symlink. Both regular files have verified content identities. A control archive with the same regular files and no symlink produces identical component rows, IDs and hashes, an empty unavailable list, and complete-within-inventory coverage. Every inspector and projection command succeeds with exit 0.

Reproduction

Use Node.js v24.22.1 from the baseline's pinned toolchain, with locked dependencies installed and the CLI already built. The verification used the prepared checkout; it did not reinstall dependencies or rebuild REA. Run this from the repository root. It creates both inputs, obtains genuine inspector Evidence, and projects it through the public CLI in an isolated HOME/XDG environment:

case_dir=$(mktemp -d)
python3 -I - "$case_dir" <<'PYTHON'
import json, os, pathlib, stat, subprocess, sys, zipfile

root = pathlib.Path(sys.argv[1]).resolve()
root.mkdir(parents=True, exist_ok=True)
env = {"PATH": os.environ["PATH"]}
for key, name in (("HOME", "home"), ("XDG_CONFIG_HOME", "config"),
                  ("XDG_CACHE_HOME", "cache"), ("XDG_DATA_HOME", "data"),
                  ("XDG_STATE_HOME", "state"), ("TMPDIR", "temp")):
    directory = root / name
    directory.mkdir(exist_ok=True)
    env[key] = str(directory)

def cli(*args):
    result = subprocess.run(["node", "scripts/rea.mjs", *args, "--format", "json"],
                            env=env, capture_output=True, text=True)
    assert result.returncode == 0, result.stderr + result.stdout
    return json.loads(result.stdout)

graphs = []
for label, linked in (("linked", True), ("plain", False)):
    package = root / (label + ".hap")
    with zipfile.ZipFile(package, "w") as archive:
        archive.writestr("module.json", '{"module":{"name":"verifier"}}')
        archive.writestr("ets/entry.abc", b"fixture bytecode")
        if linked:
            link = zipfile.ZipInfo("ets/current")
            link.create_system = 3
            link.external_attr = (stat.S_IFLNK | 0o777) << 16
            archive.writestr(link, "entry.abc")
    inspection = cli("inspect-artifact", str(package))
    (root / (label + "-inspection.json")).write_text(json.dumps(inspection))
    request = root / (label + "-input.json")
    request.write_text(json.dumps({"inventory_evidence": [inspection]}))
    evidence = cli("project-harmony-application-graph", str(request))
    (root / (label + "-projection.json")).write_text(json.dumps(evidence))
    graph = evidence["normalized_result"]
    graphs.append(graph)
    inventory = inspection["normalized_result"]["substeps"][0]["evidence"]["normalized_result"]
    print(json.dumps({"fixture": label, "inventory_coverage": inventory["coverage"],
                      "coverage": graph["coverage"],
                      "unavailable_components": graph["unavailable_components"],
                      "component_paths": [row["path"] for group in graph["components"].values() for row in group],
                      "unknown_absence_notice": any("component and runtime absence is unknown" in line for line in graph["limitations"])}))
assert graphs[0]["components"] == graphs[1]["components"]
print("Component rows, content identities, and hashes match between fixtures.")
PYTHON

Compare the two printed results. The linked input reports an unavailable ets/current and partial projection coverage, while the plain input reports none and complete projection coverage. The final assertion verifies equality of the complete regular-file component rows, including their identities and hashes.

This difference reproduced in two independently executed end-to-end HAP comparisons. The inline procedure is the second comparison. It runs four REA commands; all four exited 0.

Expected behavior

A known symlink should retain its source-inventory identity without becoming unavailable file content. With a complete container-entry inventory, identical authenticated regular-file components, and no omitted bridge candidates, adding only that symlink should leave unavailable_components empty and coverage complete-within-inventory. The unknown component/runtime absence notice should not be introduced solely by the link.

The Harmony guide distinguishes file/slice component paths from symlinks retained in source inventory. Apple's corresponding guide explicitly excludes symlinks from unavailable file components. A separate public Apple ZIP control confirmed that distinction: its link was listed in symlinks, its unavailable list stayed empty, and coverage stayed complete. This report does not require adding Apple's separate link list to Harmony or reading a link's stored target.

Impact and scope

The demonstrated failure is a false unavailable-component record and misleading coverage/absence uncertainty for an accepted synthetic HAP. The authenticated regular-file components remain correct and the original inventory retains the known entry kinds. No content loss, file mutation, execution or security boundary crossing was observed.

Impact level: P2
Assessment status: supported
Impact basis: A bounded class of successful graph projections reports unnecessary unknown content and partial coverage, which can mislead consumers interpreting completeness.
Workaround status: available
Workaround basis: Cross-check unavailable paths against source inventory occurrence kinds, which the inspector retained in this reproduction, and distinguish known symlinks from unavailable file/slice content. This helps interpret the result; it does not correct the graph's reported status.

No installable or signed HAP, HSP, App Pack, escaping link, MCP call or other host platform was tested. The fixture's bytecode-named member contains inert placeholder bytes; no valid Ark decoding or runtime behavior is claimed. Complete coverage here refers only to inventoried container entries.

REA version

Exact output of node scripts/rea.mjs --version:

6.8.0

Source baseline: 65f1c447344a423fa2785847dbcea1e9a9e2454a.

Runtime and platform

Node.js v24.22.1; macOS 27.0.1, build 26A434; darwin arm64. Python's standard-library ZIP writer generates the fixtures. No external analysis provider, Harmony SDK or device is involved.

Target and artifact context

Two generated ZIP-format HAPs contain module.json and ets/entry.abc. Only the linked variant also contains the Unix-mode symlink ets/current with stored target text entry.abc. REA inventories its kind without reading its stored target. No member is extracted or executed.

Interface

CLI

Earliest failure stage

Transport or result projection

Relevant output or evidence

The inline reproduction prints the following selected fields from the actual inspector and graph results:

{"fixture": "linked", "inventory_coverage": {"scope": "container-entries", "status": "complete"}, "coverage": {"status": "partial", "inventory_complete": true}, "unavailable_components": [{"path": "ets/current", "artifact_id": null, "sha256": null, "format": "file", "encrypted": false, "limitations": ["Archive symlink target was not read."]}], "component_paths": ["module.json", "ets/entry.abc"], "unknown_absence_notice": true}
{"fixture": "plain", "inventory_coverage": {"scope": "container-entries", "status": "complete"}, "coverage": {"status": "complete-within-inventory", "inventory_complete": true}, "unavailable_components": [], "component_paths": ["module.json", "ets/entry.abc"], "unknown_absence_notice": false}

Direct inspection of the linked inventory also confirmed entry_kind: "symlink", hash_status: "not-hashed" and artifact_id: null for ets/current; both ordinary members have entry_kind: "file" and hash_status: "verified". Equality of both graphs' component objects passed.

At the recorded baseline:

Expected source: docs/harmony-analysis.md:35–63 describes the supported workflow's output and distinguishes file/slice components from link identities retained in the source inventory. docs/apple-application-analysis.md:62–70 explicitly excludes symlinks from unavailable file components.
Failure source: src/domain/harmony/harmonyApplication.ts:84–89 collects unavailable entries without the file/slice restriction used for packaging entries; :110–113 adds the unknown-absence notice and :146–150 makes a nonempty unavailable list force partial coverage.
Shared producer-to-projection path: src/domain/mobileApplicationGraphProjection.ts:135–153 retains the known entry kind while constructing an unavailable record for a null content identity; :176–183 already selects only files/slices for authenticated component classification.
Comparison: src/domain/apple/appleApplication.ts:126–134 filters unavailable entries by file/slice kind, and tests/boundary/filesystem/appleApplication.test.ts:297–311 covers a symlink keeping coverage complete. The separate Apple CLI control in this verification also passed.

Evidence provenance: observed public CLI behavior, corroborated by source inspection. Local verification: independently reproduced. Reproduction completeness: complete for the synthetic HAP pair.

Typed outcome or recovery expected

For the linked fixture, retain both current component objects, return unavailable_components: [], and report coverage: {"status": "complete-within-inventory", "inventory_complete": true}. Preserve the link occurrence and its existing limitation in source inventory. Do not fabricate a link content hash, follow the link, or remove legitimate partial coverage caused by genuinely unread files or omitted candidates.

Analysis or suspected cause

Source-derived explanation: the shared occurrence representation preserves kind alongside unavailable identity, but Harmony collects the unavailable values before applying its file/slice selection. Thus its correct exclusion of links from authenticated components does not extend to unavailable components. Apple's filter handles the same distinction. The public producer/projection comparison establishes that this is reachable without constructing or modifying Evidence manually.

Additional context

Duplicate searches included open and closed issues, issue comments, and open, closed and merged PRs for Harmony, coverage, symlink and unavailable components. The adjacent reports have different closure signals:

#4688 and PR #4692 fixed Apple dropping unread regular files. The Apple symlink exclusion works in this verification; this is not an observed regression of that fix.
#5748 concerns path-derived packaging hints and App Pack child paths lost for encrypted regular files, whose partial coverage is appropriate.
#4489 and open PR #4968 concern authenticated unclassified files and N-API declarations. That PR's patch leaves Harmony's unavailable list unchanged, so its completion would not remove the false link row or coverage downgrade.
#3349 and PR #3830 excluded directories/links from authenticated code components; their documented scope preserves existing unavailable-component handling. This reproduction already has the correct code component lists.
#6751 concerns the inventory reader misidentifying mode-only ZIP links. Here the reader correctly identifies a symlink with permission bits.

No matching report or PR fixing the demonstrated unavailable-link behavior was found. No last-known-good version or regression onset was established.

Originally created by @coygeek on GitHub (Oct 27, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/7545 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior After inspecting a generated HAP, `node scripts/rea.mjs project-harmony-application-graph <input.json> --format json` accepts the inspector's Evidence but reports the known ZIP symlink `ets/current` in `unavailable_components`. It also returns `coverage.status: "partial"` and a limitation saying component and runtime absence is unknown. The input inventory reports complete container-entry coverage and identifies that occurrence as a symlink. Both regular files have verified content identities. A control archive with the same regular files and no symlink produces identical component rows, IDs and hashes, an empty unavailable list, and `complete-within-inventory` coverage. Every inspector and projection command succeeds with exit 0. ### Reproduction 1. Use Node.js v24.22.1 from the baseline's pinned toolchain, with locked dependencies installed and the CLI already built. The verification used the prepared checkout; it did not reinstall dependencies or rebuild REA. Run this from the repository root. It creates both inputs, obtains genuine inspector Evidence, and projects it through the public CLI in an isolated HOME/XDG environment: ```sh case_dir=$(mktemp -d) python3 -I - "$case_dir" <<'PYTHON' import json, os, pathlib, stat, subprocess, sys, zipfile root = pathlib.Path(sys.argv[1]).resolve() root.mkdir(parents=True, exist_ok=True) env = {"PATH": os.environ["PATH"]} for key, name in (("HOME", "home"), ("XDG_CONFIG_HOME", "config"), ("XDG_CACHE_HOME", "cache"), ("XDG_DATA_HOME", "data"), ("XDG_STATE_HOME", "state"), ("TMPDIR", "temp")): directory = root / name directory.mkdir(exist_ok=True) env[key] = str(directory) def cli(*args): result = subprocess.run(["node", "scripts/rea.mjs", *args, "--format", "json"], env=env, capture_output=True, text=True) assert result.returncode == 0, result.stderr + result.stdout return json.loads(result.stdout) graphs = [] for label, linked in (("linked", True), ("plain", False)): package = root / (label + ".hap") with zipfile.ZipFile(package, "w") as archive: archive.writestr("module.json", '{"module":{"name":"verifier"}}') archive.writestr("ets/entry.abc", b"fixture bytecode") if linked: link = zipfile.ZipInfo("ets/current") link.create_system = 3 link.external_attr = (stat.S_IFLNK | 0o777) << 16 archive.writestr(link, "entry.abc") inspection = cli("inspect-artifact", str(package)) (root / (label + "-inspection.json")).write_text(json.dumps(inspection)) request = root / (label + "-input.json") request.write_text(json.dumps({"inventory_evidence": [inspection]})) evidence = cli("project-harmony-application-graph", str(request)) (root / (label + "-projection.json")).write_text(json.dumps(evidence)) graph = evidence["normalized_result"] graphs.append(graph) inventory = inspection["normalized_result"]["substeps"][0]["evidence"]["normalized_result"] print(json.dumps({"fixture": label, "inventory_coverage": inventory["coverage"], "coverage": graph["coverage"], "unavailable_components": graph["unavailable_components"], "component_paths": [row["path"] for group in graph["components"].values() for row in group], "unknown_absence_notice": any("component and runtime absence is unknown" in line for line in graph["limitations"])})) assert graphs[0]["components"] == graphs[1]["components"] print("Component rows, content identities, and hashes match between fixtures.") PYTHON ``` 2. Compare the two printed results. The linked input reports an unavailable `ets/current` and partial projection coverage, while the plain input reports none and complete projection coverage. The final assertion verifies equality of the complete regular-file component rows, including their identities and hashes. This difference reproduced in two independently executed end-to-end HAP comparisons. The inline procedure is the second comparison. It runs four REA commands; all four exited 0. ### Expected behavior A known symlink should retain its source-inventory identity without becoming unavailable file content. With a complete container-entry inventory, identical authenticated regular-file components, and no omitted bridge candidates, adding only that symlink should leave `unavailable_components` empty and coverage `complete-within-inventory`. The unknown component/runtime absence notice should not be introduced solely by the link. The Harmony guide distinguishes file/slice component paths from symlinks retained in source inventory. Apple's corresponding guide explicitly excludes symlinks from unavailable file components. A separate public Apple ZIP control confirmed that distinction: its link was listed in `symlinks`, its unavailable list stayed empty, and coverage stayed complete. This report does not require adding Apple's separate link list to Harmony or reading a link's stored target. ### Impact and scope The demonstrated failure is a false unavailable-component record and misleading coverage/absence uncertainty for an accepted synthetic HAP. The authenticated regular-file components remain correct and the original inventory retains the known entry kinds. No content loss, file mutation, execution or security boundary crossing was observed. - Impact level: P2 - Assessment status: supported - Impact basis: A bounded class of successful graph projections reports unnecessary unknown content and partial coverage, which can mislead consumers interpreting completeness. - Workaround status: available - Workaround basis: Cross-check unavailable paths against source inventory occurrence kinds, which the inspector retained in this reproduction, and distinguish known symlinks from unavailable file/slice content. This helps interpret the result; it does not correct the graph's reported status. No installable or signed HAP, HSP, App Pack, escaping link, MCP call or other host platform was tested. The fixture's bytecode-named member contains inert placeholder bytes; no valid Ark decoding or runtime behavior is claimed. Complete coverage here refers only to inventoried container entries. ### REA version Exact output of `node scripts/rea.mjs --version`: ```text 6.8.0 ``` Source baseline: `65f1c447344a423fa2785847dbcea1e9a9e2454a`. ### Runtime and platform Node.js v24.22.1; macOS 27.0.1, build 26A434; darwin arm64. Python's standard-library ZIP writer generates the fixtures. No external analysis provider, Harmony SDK or device is involved. ### Target and artifact context Two generated ZIP-format HAPs contain `module.json` and `ets/entry.abc`. Only the linked variant also contains the Unix-mode symlink `ets/current` with stored target text `entry.abc`. REA inventories its kind without reading its stored target. No member is extracted or executed. ### Interface CLI ### Earliest failure stage Transport or result projection ### Relevant output or evidence The inline reproduction prints the following selected fields from the actual inspector and graph results: ```json {"fixture": "linked", "inventory_coverage": {"scope": "container-entries", "status": "complete"}, "coverage": {"status": "partial", "inventory_complete": true}, "unavailable_components": [{"path": "ets/current", "artifact_id": null, "sha256": null, "format": "file", "encrypted": false, "limitations": ["Archive symlink target was not read."]}], "component_paths": ["module.json", "ets/entry.abc"], "unknown_absence_notice": true} {"fixture": "plain", "inventory_coverage": {"scope": "container-entries", "status": "complete"}, "coverage": {"status": "complete-within-inventory", "inventory_complete": true}, "unavailable_components": [], "component_paths": ["module.json", "ets/entry.abc"], "unknown_absence_notice": false} ``` Direct inspection of the linked inventory also confirmed `entry_kind: "symlink"`, `hash_status: "not-hashed"` and `artifact_id: null` for `ets/current`; both ordinary members have `entry_kind: "file"` and `hash_status: "verified"`. Equality of both graphs' component objects passed. At the recorded baseline: - Expected source: `docs/harmony-analysis.md:35–63` describes the supported workflow's output and distinguishes file/slice components from link identities retained in the source inventory. `docs/apple-application-analysis.md:62–70` explicitly excludes symlinks from unavailable file components. - Failure source: `src/domain/harmony/harmonyApplication.ts:84–89` collects unavailable entries without the file/slice restriction used for packaging entries; `:110–113` adds the unknown-absence notice and `:146–150` makes a nonempty unavailable list force partial coverage. - Shared producer-to-projection path: `src/domain/mobileApplicationGraphProjection.ts:135–153` retains the known entry kind while constructing an unavailable record for a null content identity; `:176–183` already selects only files/slices for authenticated component classification. - Comparison: `src/domain/apple/appleApplication.ts:126–134` filters unavailable entries by file/slice kind, and `tests/boundary/filesystem/appleApplication.test.ts:297–311` covers a symlink keeping coverage complete. The separate Apple CLI control in this verification also passed. Evidence provenance: observed public CLI behavior, corroborated by source inspection. Local verification: independently reproduced. Reproduction completeness: complete for the synthetic HAP pair. ### Typed outcome or recovery expected For the linked fixture, retain both current component objects, return `unavailable_components: []`, and report `coverage: {"status": "complete-within-inventory", "inventory_complete": true}`. Preserve the link occurrence and its existing limitation in source inventory. Do not fabricate a link content hash, follow the link, or remove legitimate partial coverage caused by genuinely unread files or omitted candidates. ### Analysis or suspected cause Source-derived explanation: the shared occurrence representation preserves `kind` alongside unavailable identity, but Harmony collects the unavailable values before applying its file/slice selection. Thus its correct exclusion of links from authenticated components does not extend to unavailable components. Apple's filter handles the same distinction. The public producer/projection comparison establishes that this is reachable without constructing or modifying Evidence manually. ### Additional context Duplicate searches included open and closed issues, issue comments, and open, closed and merged PRs for Harmony, coverage, symlink and unavailable components. The adjacent reports have different closure signals: - [#4688](https://github.com/morluto/rea/issues/4688) and [PR #4692](https://github.com/morluto/rea/pull/4692) fixed Apple dropping unread regular files. The Apple symlink exclusion works in this verification; this is not an observed regression of that fix. - [#5748](https://github.com/morluto/rea/issues/5748) concerns path-derived packaging hints and App Pack child paths lost for encrypted regular files, whose partial coverage is appropriate. - [#4489](https://github.com/morluto/rea/issues/4489) and open [PR #4968](https://github.com/morluto/rea/pull/4968) concern authenticated unclassified files and N-API declarations. That PR's patch leaves Harmony's unavailable list unchanged, so its completion would not remove the false link row or coverage downgrade. - [#3349](https://github.com/morluto/rea/issues/3349) and [PR #3830](https://github.com/morluto/rea/pull/3830) excluded directories/links from authenticated code components; their documented scope preserves existing unavailable-component handling. This reproduction already has the correct code component lists. - [#6751](https://github.com/morluto/rea/issues/6751) concerns the inventory reader misidentifying mode-only ZIP links. Here the reader correctly identifies a symlink with permission bits. No matching report or PR fixing the demonstrated unavailable-link behavior was found. No last-known-good version or regression onset was established.

bsakaguchi closed this issue

2026-10-31 00:50:13 +00:00

bsakaguchi referenced this issue

2026-10-31 01:13:11 +00:00

[PR #4486] [CLOSED] fix(cutter): admit bridge descriptors without blocking on pipes #6271

bsakaguchi referenced this issue

2026-10-31 01:13:12 +00:00

[PR #4499] [MERGED] fix(cutter): cancel discovery probes and queued work #6275