morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #5505] [Bug] Rust report-budget refusals recommend recovering Evidence that was never produced #3349

New issue

Closed

opened 2026-10-31 00:39:25 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-31 00:39:25 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/5505

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

CLI

Observed behavior

node scripts/rea.mjs inspect-rust-binary <artifact> --json correctly refuses a dependency graph above its fixed 8 MiB complete-report allowance, but recommends retrieving retained Evidence, using analysis views, exporting complete session evidence, or using complete CLI JSON. The failed inspection produces no completed report to retrieve, Rust Evidence does not support analysis views, and the request already uses complete CLI JSON.

Reproduction

An independent verifier ran this exact script end to end from the baseline source checkout with dependencies and the built CLI prepared. It generates thin Mach-O executables with clang without executing them, isolates configuration/state, runs a successful small control and two oversized complete-JSON requests, then requests a summary view of the successful control Evidence.

python3 - <<'PYTHON'
import hashlib, json, os, pathlib, subprocess, tempfile, zlib
work = pathlib.Path(tempfile.mkdtemp(prefix="rust-report-check-"))
env = {k: v for k, v in os.environ.items()
       if not k.startswith(("REA_", "HOPPER_", "GHIDRA_")) and k != "NODE_OPTIONS"}
for key, name in [("HOME", "home"), ("XDG_CONFIG_HOME", "config"),
                  ("XDG_DATA_HOME", "data"), ("XDG_CACHE_HOME", "cache"),
                  ("XDG_STATE_HOME", "state"), ("TMPDIR", "temp")]:
    directory = work / name
    directory.mkdir()
    env[key] = str(directory)
def run(label, arguments):
    result = subprocess.run(["node", "scripts/rea.mjs", *arguments, "--json"],
                            env=env, capture_output=True, text=True, timeout=60)
    (work / (label + ".json")).write_text(result.stdout)
    (work / (label + ".stderr")).write_text(result.stderr)
    value = json.loads(result.stdout)
    print(json.dumps({"case": label, "exit": result.returncode,
                      "code": value.get("code"), "message": value.get("message"),
                      "details": value.get("details"),
                      "remediation": value.get("remediation")}))
    return value
source = work / "main.c"
source.write_text("int main(void){return 0;}\n")
for count in [1, 450000]:
    document = {"packages": [{"name": "sample", "version": "1.0.0",
        "source": "local", "dependencies": [0] * count, "root": True}]}
    raw = json.dumps(document, separators=(",", ":")).encode()
    payload = zlib.compress(raw, 0)
    section = work / (str(count) + ".dep")
    section.write_bytes(payload)
    artifact = work / ("app-" + str(count))
    subprocess.run(["clang", str(source), "-o", str(artifact),
                    "-Wl,-sectcreate,__DATA,.dep-v0," + str(section)],
                   env=env, capture_output=True, text=True, check=True, timeout=60)
    digest = hashlib.sha256(artifact.read_bytes()).hexdigest()
    print(json.dumps({"edges": count, "input_bytes": artifact.stat().st_size,
                      "compressed_bytes": len(payload), "inflated_bytes": len(raw)}))
    for attempt in range(1 if count == 1 else 2):
        value = run(str(count) + "-" + str(attempt),
                    ["inspect-rust-binary", str(artifact)])
        if count == 1:
            control = value
            print(json.dumps({"returned_edges": len(value["normalized_result"]
                ["auditable_metadata"]["dependency_edges"])}))
    assert hashlib.sha256(artifact.read_bytes()).hexdigest() == digest
request = work / "summary-request.json"
request.write_text(json.dumps({"source": {"kind": "inline", "evidence": control},
                               "view": {"kind": "summary"}}))
run("rust-summary", ["inspect-analysis-view", str(request)])
PYTHON

The one-edge control succeeds. Both 450,000-edge requests exit 1 with the recovery action below. The control Evidence's summary view exits 1 with unsupported_target, explicitly stating that parent operation inspect_rust_binary does not support analysis views. The verifier also replayed the script, with the same results.

Expected behavior

Preserve the documented fixed report-budget refusal and typed failure without partial success, but give recovery guidance applicable to a report that was never completed. Do not imply that retained-report retrieval, Rust analysis views, or complete CLI JSON can recover this failed graph.

docs/rust-binaries.md:47-72 documents independent input, section, inflation and complete-report bounds, including count-based graph admission before expansion. src/contracts/rust/rustToolContracts.ts:15-16 promises typed failures rather than partial success. docs/mcp-contracts.md:102-115 describes a different case: actual MCP delivery-budget failures after complete Evidence exists. This expectation follows from those documented distinctions.

Impact and scope

P3 diagnostic wording defect. Two oversized CLI attempts and the subsequent script replay returned the same inapplicable advice. The intentional refusal, numeric allowance and non-retryable status remain correct. No incorrect successful analysis, data loss or consequential behavior change caused by the advice was established. Frequency in real cargo-auditable artifacts is unknown. Live MCP was not exercised.

Callers can identify the fixed allowance from the message and maximum_output_bytes and avoid the inapplicable recovery calls. A smaller graph succeeds, but selecting another artifact does not recover the original oversized graph.

REA version

6.7.0; source commit 074768901f3cdc8205b61773fb1faf6f6c7b3ca8.

Runtime and platform

Node v26.12.0; Darwin arm64; Apple clang 21.0.0. No Hopper, Ghidra or other analysis engine was used.

Target and artifact context

Synthetic thin Mach-O executables with a zlib dependency document containing one package and either one or 450,000 dependency indices referencing that package. The oversized selected artifact was 925,000 bytes, its compressed section 900,177 bytes and inflated JSON 900,096 bytes, below the earlier documented bounds. Input SHA-256 digests remained unchanged. No target was executed.

Interface

CLI

Earliest failure stage

Native analysis or reconstruction

Relevant output or evidence

Both oversized requests exited 1 with empty stderr and this observed diagnostic subset:

{
  "code": "resource_constraint",
  "category": "resource_constraint",
  "retryable": false,
  "remediation": {
    "action": "Discover retained records with get_evidence_bundle and detail: summary, then inspect compatible records through inspect_analysis_view using an exact Evidence reference. Export complete retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. The connection remains usable."
  },
  "details": {
    "operation": "inspect_rust_binary",
    "resource": "transport",
    "reported_limits": { "maximum_output_bytes": 8388608 }
  }
}

The message identifies the dependency graph exceeding the complete-report byte budget before expansion. Neither failed result contained an Evidence ID, export reference or partial observation. The small control exited 0 and returned one package, one dependency edge and the self-cycle index. The script above supplies the observed CLI evidence; it does not establish live MCP retention behavior.

Typed outcome or recovery expected

Keep resource_constraint, the numeric 8 MiB allowance, non-retryable status and no-partial-success behavior. Explain the complete-report limit with applicable recovery guidance. Preserve delivery-recovery advice for genuine MCP delivery failures after completed Evidence; that is a proposed regression control, not an executed MCP test here.

Analysis or suspected cause

Source-derived reasoning at the tested revision: src/rust/RustAuditableReader.ts:281-305 refuses the graph before expansion. src/rust/RustBinaryProvider.ts:181-190 maps that failure without specific remediation, so src/domain/analysisErrorPresentation.ts:67-78 supplies generic transport-delivery advice. src/application/rust/RustBinaryService.ts:69-81 returns the failure before Evidence creation. The shared MCP service and handler (src/server/registerRustTools.ts:18-25, src/server/contractToolHandler.ts:34-52) suggest the same failure is not recorded as successful or partial Evidence, but this MCP behavior was not runtime-tested.

The executed unsupported summary-view control agrees with the projector dispatch in src/domain/analysisView/analysisView.ts:833-849 and unsupported-parent fallback.

Additional context

Prepared with AI assistance and independently reproduced through the supported built CLI. Submission checks fetched upstream main at c374c2fe7290b914d73d9e27af38b498753ec593. No later commit changed the Rust refusal, provider mapping, error presentation, service or handler; changes in the other cited files concern native function matching, absolute-path schemas and Ghidra. Applicability to current main is source-checked, not runtime-reproduced there.

Related but separate: #5456 concerns SQLite's worker-reply guard, #5459 concerns analysis-view representation accounting, and #5460 concerns IL2CPP's encoded-report guard. Their producer-specific corrections can leave this Rust graph refusal unchanged. #5405 concerns the earlier Rust/IL2CPP input-size admission guard; this fixture passes that boundary. #5504 concerns compiler-path hash capacity and retry advice, a different producer outside this report.

NOT_ENOUGH_INFO: no last-known-good version or regression boundary was established. Runtime MCP delivery and ledger behavior remain source-derived.

Originally created by @coygeek on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/5505 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area CLI ### Observed behavior `node scripts/rea.mjs inspect-rust-binary <artifact> --json` correctly refuses a dependency graph above its fixed 8 MiB complete-report allowance, but recommends retrieving retained Evidence, using analysis views, exporting complete session evidence, or using complete CLI JSON. The failed inspection produces no completed report to retrieve, Rust Evidence does not support analysis views, and the request already uses complete CLI JSON. ### Reproduction An independent verifier ran this exact script end to end from the baseline source checkout with dependencies and the built CLI prepared. It generates thin Mach-O executables with clang without executing them, isolates configuration/state, runs a successful small control and two oversized complete-JSON requests, then requests a summary view of the successful control Evidence. ```sh python3 - <<'PYTHON' import hashlib, json, os, pathlib, subprocess, tempfile, zlib work = pathlib.Path(tempfile.mkdtemp(prefix="rust-report-check-")) env = {k: v for k, v in os.environ.items() if not k.startswith(("REA_", "HOPPER_", "GHIDRA_")) and k != "NODE_OPTIONS"} for key, name in [("HOME", "home"), ("XDG_CONFIG_HOME", "config"), ("XDG_DATA_HOME", "data"), ("XDG_CACHE_HOME", "cache"), ("XDG_STATE_HOME", "state"), ("TMPDIR", "temp")]: directory = work / name directory.mkdir() env[key] = str(directory) def run(label, arguments): result = subprocess.run(["node", "scripts/rea.mjs", *arguments, "--json"], env=env, capture_output=True, text=True, timeout=60) (work / (label + ".json")).write_text(result.stdout) (work / (label + ".stderr")).write_text(result.stderr) value = json.loads(result.stdout) print(json.dumps({"case": label, "exit": result.returncode, "code": value.get("code"), "message": value.get("message"), "details": value.get("details"), "remediation": value.get("remediation")})) return value source = work / "main.c" source.write_text("int main(void){return 0;}\n") for count in [1, 450000]: document = {"packages": [{"name": "sample", "version": "1.0.0", "source": "local", "dependencies": [0] * count, "root": True}]} raw = json.dumps(document, separators=(",", ":")).encode() payload = zlib.compress(raw, 0) section = work / (str(count) + ".dep") section.write_bytes(payload) artifact = work / ("app-" + str(count)) subprocess.run(["clang", str(source), "-o", str(artifact), "-Wl,-sectcreate,__DATA,.dep-v0," + str(section)], env=env, capture_output=True, text=True, check=True, timeout=60) digest = hashlib.sha256(artifact.read_bytes()).hexdigest() print(json.dumps({"edges": count, "input_bytes": artifact.stat().st_size, "compressed_bytes": len(payload), "inflated_bytes": len(raw)})) for attempt in range(1 if count == 1 else 2): value = run(str(count) + "-" + str(attempt), ["inspect-rust-binary", str(artifact)]) if count == 1: control = value print(json.dumps({"returned_edges": len(value["normalized_result"] ["auditable_metadata"]["dependency_edges"])})) assert hashlib.sha256(artifact.read_bytes()).hexdigest() == digest request = work / "summary-request.json" request.write_text(json.dumps({"source": {"kind": "inline", "evidence": control}, "view": {"kind": "summary"}})) run("rust-summary", ["inspect-analysis-view", str(request)]) PYTHON ``` The one-edge control succeeds. Both 450,000-edge requests exit 1 with the recovery action below. The control Evidence's summary view exits 1 with `unsupported_target`, explicitly stating that parent operation `inspect_rust_binary` does not support analysis views. The verifier also replayed the script, with the same results. ### Expected behavior Preserve the documented fixed report-budget refusal and typed failure without partial success, but give recovery guidance applicable to a report that was never completed. Do not imply that retained-report retrieval, Rust analysis views, or complete CLI JSON can recover this failed graph. `docs/rust-binaries.md:47-72` documents independent input, section, inflation and complete-report bounds, including count-based graph admission before expansion. `src/contracts/rust/rustToolContracts.ts:15-16` promises typed failures rather than partial success. `docs/mcp-contracts.md:102-115` describes a different case: actual MCP delivery-budget failures after complete Evidence exists. This expectation follows from those documented distinctions. ### Impact and scope P3 diagnostic wording defect. Two oversized CLI attempts and the subsequent script replay returned the same inapplicable advice. The intentional refusal, numeric allowance and non-retryable status remain correct. No incorrect successful analysis, data loss or consequential behavior change caused by the advice was established. Frequency in real cargo-auditable artifacts is unknown. Live MCP was not exercised. Callers can identify the fixed allowance from the message and `maximum_output_bytes` and avoid the inapplicable recovery calls. A smaller graph succeeds, but selecting another artifact does not recover the original oversized graph. ### REA version 6.7.0; source commit `074768901f3cdc8205b61773fb1faf6f6c7b3ca8`. ### Runtime and platform Node v26.12.0; Darwin arm64; Apple clang 21.0.0. No Hopper, Ghidra or other analysis engine was used. ### Target and artifact context Synthetic thin Mach-O executables with a zlib dependency document containing one package and either one or 450,000 dependency indices referencing that package. The oversized selected artifact was 925,000 bytes, its compressed section 900,177 bytes and inflated JSON 900,096 bytes, below the earlier documented bounds. Input SHA-256 digests remained unchanged. No target was executed. ### Interface CLI ### Earliest failure stage Native analysis or reconstruction ### Relevant output or evidence Both oversized requests exited 1 with empty stderr and this observed diagnostic subset: ```json { "code": "resource_constraint", "category": "resource_constraint", "retryable": false, "remediation": { "action": "Discover retained records with get_evidence_bundle and detail: summary, then inspect compatible records through inspect_analysis_view using an exact Evidence reference. Export complete retained session evidence through export_evidence_bundle to a caller-selected path, or use complete CLI JSON output. The connection remains usable." }, "details": { "operation": "inspect_rust_binary", "resource": "transport", "reported_limits": { "maximum_output_bytes": 8388608 } } } ``` The message identifies the dependency graph exceeding the complete-report byte budget before expansion. Neither failed result contained an Evidence ID, export reference or partial observation. The small control exited 0 and returned one package, one dependency edge and the self-cycle index. The script above supplies the observed CLI evidence; it does not establish live MCP retention behavior. ### Typed outcome or recovery expected Keep `resource_constraint`, the numeric 8 MiB allowance, non-retryable status and no-partial-success behavior. Explain the complete-report limit with applicable recovery guidance. Preserve delivery-recovery advice for genuine MCP delivery failures after completed Evidence; that is a proposed regression control, not an executed MCP test here. ### Analysis or suspected cause Source-derived reasoning at the tested revision: `src/rust/RustAuditableReader.ts:281-305` refuses the graph before expansion. `src/rust/RustBinaryProvider.ts:181-190` maps that failure without specific remediation, so `src/domain/analysisErrorPresentation.ts:67-78` supplies generic transport-delivery advice. `src/application/rust/RustBinaryService.ts:69-81` returns the failure before Evidence creation. The shared MCP service and handler (`src/server/registerRustTools.ts:18-25`, `src/server/contractToolHandler.ts:34-52`) suggest the same failure is not recorded as successful or partial Evidence, but this MCP behavior was not runtime-tested. The executed unsupported summary-view control agrees with the projector dispatch in `src/domain/analysisView/analysisView.ts:833-849` and unsupported-parent fallback. ### Additional context Prepared with AI assistance and independently reproduced through the supported built CLI. Submission checks fetched upstream main at `c374c2fe7290b914d73d9e27af38b498753ec593`. No later commit changed the Rust refusal, provider mapping, error presentation, service or handler; changes in the other cited files concern native function matching, absolute-path schemas and Ghidra. Applicability to current main is source-checked, not runtime-reproduced there. Related but separate: #5456 concerns SQLite's worker-reply guard, #5459 concerns analysis-view representation accounting, and #5460 concerns IL2CPP's encoded-report guard. Their producer-specific corrections can leave this Rust graph refusal unchanged. #5405 concerns the earlier Rust/IL2CPP input-size admission guard; this fixture passes that boundary. #5504 concerns compiler-path hash capacity and retry advice, a different producer outside this report. NOT_ENOUGH_INFO: no last-known-good version or regression boundary was established. Runtime MCP delivery and ledger behavior remain source-derived.

bsakaguchi closed this issue

2026-10-31 00:39:26 +00:00

bsakaguchi commented

2026-10-31 00:39:27 +00:00

Author

Owner

@matbrgz commented on GitHub (Oct 20, 2026):

Cross-referencing with sibling budget and capacity refusal reports in Cluster AJ: #5504 ([Bug] Rust compiler-hash capacity refusals incorrectly tell callers to retry), #5506, #5456, #5459, and #5460. All isolate misleading remediation text in provider budget refusals (advising retries or recovering nonexistent Evidence).

Consolidated under Cluster AJ (Misleading Recovery Remediation on Capacity & Budget Refusals) in tracker #4330.

@matbrgz commented on GitHub (Oct 20, 2026): Cross-referencing with sibling budget and capacity refusal reports in **Cluster AJ**: #5504 (`[Bug] Rust compiler-hash capacity refusals incorrectly tell callers to retry`), #5506, #5456, #5459, and #5460. All isolate misleading remediation text in provider budget refusals (advising retries or recovering nonexistent Evidence). Consolidated under **Cluster AJ (Misleading Recovery Remediation on Capacity & Budget Refusals)** in tracker #4330.

bsakaguchi commented

2026-10-31 00:39:27 +00:00

Author

Owner

@morluto commented on GitHub (Oct 21, 2026):

Closing as fixed on main by github.com/morluto/rea@ef59174c1e, retained at 67a6477f01.

Rust complete-report refusal now uses report-specific recovery: no completed report/Evidence exists, and complete CLI JSON does not raise the allowance. The producer assignment and shared projection tests cover the correction; the exact full-provider report-overflow remediation branch was not rerun or separately asserted in this audit.

This audit inspected the current producer, regression source, issue discussion and main ancestry. Tests/runtime reproductions were not rerun, and no additional real-provider or platform coverage is implied.

@morluto commented on GitHub (Oct 21, 2026): Closing as fixed on main by https://github.com/morluto/rea/commit/ef59174c1edd6ed9adb0722b86a5b9a57217e310, retained at 67a6477f011ae358fc333e5a6e26c86d932654ed. Rust complete-report refusal now uses report-specific recovery: no completed report/Evidence exists, and complete CLI JSON does not raise the allowance. The producer assignment and shared projection tests cover the correction; the exact full-provider report-overflow remediation branch was not rerun or separately asserted in this audit. This audit inspected the current producer, regression source, issue discussion and main ancestry. Tests/runtime reproductions were not rerun, and no additional real-provider or platform coverage is implied.

bsakaguchi referenced this issue

2026-10-31 01:10:33 +00:00

[PR #3830] [CLOSED] fix(mobile): exclude directory entries from application components #5931

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#3349

No description provided.

Rows
Columns