morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3490] [Feature] extract_artifact: choose the destination and extract a subset of entries #1302

New issue

Open

opened 2026-10-16 21:03:56 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-16 21:03:56 +00:00

Owner

Originally created by @Rinkia on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3490

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

I wanted one file from an APK, lib/arm64-v8a/libzxingcpp_android.so, to pass to Ghidra. Two things got in the way:

Destination. rea extract-artifact has no destination option. Output always goes to os.tmpdir()/rea-extracted-<uuid> (src/application/artifacts/ArtifactExtractionDestination.ts). The only way to move it is setting TMPDIR, which isn't documented: the extract_artifact section in docs/mcp-contracts.md doesn't say where files go or that they're left there afterwards (cleanup.attempted: false). In a container, or any setup where the caller's filesystem differs from REA's, the default location may be unreachable.
All-or-nothing. Extraction materializes every regular file. With TMPDIR on a case-insensitive filesystem (the default on Windows and macOS; here a Windows bind mount), an obfuscated APK fails entirely:
```
artifact_operation_failed: Destination filesystem cannot store both res/2f.xml and res/2F.xml; inventory retains both logical names.
```
The refusal itself is correct and clearly worded, and it left no partial files behind. But the native library I needed has no case conflict, and there's no way to ask for just that entry, so the result is zero files.

Target: F-Droid de.markusfisch.android.binaryeye versionCode 180, sha256 568cec6598dbf42fbde9071af8747071ffea798473016a3d77b6ddb3bff944f1. R8/AAPT2 resource shortening routinely produces names that differ only by case, so many release APKs will hit this on Windows/macOS.

Use cases

Android: pull the native libraries for one ABI (lib/arm64-v8a/*.so) out of an APK for native analysis.
Any host with a case-insensitive default filesystem.
Containers and remote MCP setups, where the caller needs files in a mounted, chosen directory.

Desired outcome and success criteria

An explicit destination parameter (CLI flag and MCP input), with the current temp-directory behaviour kept as the default.
A selection parameter (path glob, logical paths, or occurrence ids from the inventory) that materializes only matching entries, still under the existing containment, integrity and collision rules.
Docs stating the default location, that the tree persists after the call, and that TMPDIR controls the default.
Success: on Windows, extracting lib/arm64-v8a/*.so from the APK above succeeds while the full extraction still reports the res/2f.xml collision.

Expected interaction with targets

Read existing files or captured data; write or change files (the extraction output only).

Verification available so far

I have a prototype or observed results.

Verification details

rea-agents@6.4.0 in a Linux x64 container on Windows 11 / Docker Desktop:

Default run: output_root: /tmp/rea-extracted-<uuid>, containment_verified: true, cleanup: {"attempted":false,...}.
TMPDIR pointing at a Windows bind mount: fails with the collision error above, and the destination is left empty.
TMPDIR on the container's own /tmp (case-sensitive): exit 0, and all 12 lib/*/*.so files are present.

Alternatives considered

Setting TMPDIR to a case-sensitive location. It works, but it's undocumented and needs such a filesystem to exist.
Extracting with unzip outside REA. That loses REA's containment, integrity and provenance records.

Possible direction

Reuse the extraction manifest's selected_occurrence_ids, which already exists in the result, as the input filter. Then a caller can list first and extract the chosen occurrences.

References

src/application/artifacts/ArtifactExtractionDestination.ts
docs/mcp-contracts.md (Integrity record-and-continue / extract_artifact)

Originally created by @Rinkia on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3490 ### Before submitting - [x] I searched the open and closed issues and pull requests for a related proposal. - [x] I understand the contribution scope must be reviewed before integration; open questions are welcome here. - [x] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation I wanted one file from an APK, `lib/arm64-v8a/libzxingcpp_android.so`, to pass to Ghidra. Two things got in the way: 1. **Destination.** `rea extract-artifact` has no destination option. Output always goes to `os.tmpdir()/rea-extracted-<uuid>` (`src/application/artifacts/ArtifactExtractionDestination.ts`). The only way to move it is setting `TMPDIR`, which isn't documented: the `extract_artifact` section in `docs/mcp-contracts.md` doesn't say where files go or that they're left there afterwards (`cleanup.attempted: false`). In a container, or any setup where the caller's filesystem differs from REA's, the default location may be unreachable. 2. **All-or-nothing.** Extraction materializes every regular file. With `TMPDIR` on a case-insensitive filesystem (the default on Windows and macOS; here a Windows bind mount), an obfuscated APK fails entirely: ```text artifact_operation_failed: Destination filesystem cannot store both res/2f.xml and res/2F.xml; inventory retains both logical names. ``` The refusal itself is correct and clearly worded, and it left no partial files behind. But the native library I needed has no case conflict, and there's no way to ask for just that entry, so the result is zero files. Target: F-Droid `de.markusfisch.android.binaryeye` versionCode 180, sha256 `568cec6598dbf42fbde9071af8747071ffea798473016a3d77b6ddb3bff944f1`. R8/AAPT2 resource shortening routinely produces names that differ only by case, so many release APKs will hit this on Windows/macOS. ### Use cases - Android: pull the native libraries for one ABI (`lib/arm64-v8a/*.so`) out of an APK for native analysis. - Any host with a case-insensitive default filesystem. - Containers and remote MCP setups, where the caller needs files in a mounted, chosen directory. ### Desired outcome and success criteria - An explicit destination parameter (CLI flag and MCP input), with the current temp-directory behaviour kept as the default. - A selection parameter (path glob, logical paths, or occurrence ids from the inventory) that materializes only matching entries, still under the existing containment, integrity and collision rules. - Docs stating the default location, that the tree persists after the call, and that `TMPDIR` controls the default. - Success: on Windows, extracting `lib/arm64-v8a/*.so` from the APK above succeeds while the full extraction still reports the `res/2f.xml` collision. ### Expected interaction with targets Read existing files or captured data; write or change files (the extraction output only). ### Verification available so far I have a prototype or observed results. ### Verification details `rea-agents@6.4.0` in a Linux x64 container on Windows 11 / Docker Desktop: - Default run: `output_root: /tmp/rea-extracted-<uuid>`, `containment_verified: true`, `cleanup: {"attempted":false,...}`. - `TMPDIR` pointing at a Windows bind mount: fails with the collision error above, and the destination is left empty. - `TMPDIR` on the container's own `/tmp` (case-sensitive): exit 0, and all 12 `lib/*/*.so` files are present. ### Alternatives considered - Setting `TMPDIR` to a case-sensitive location. It works, but it's undocumented and needs such a filesystem to exist. - Extracting with `unzip` outside REA. That loses REA's containment, integrity and provenance records. ### Possible direction Reuse the extraction manifest's `selected_occurrence_ids`, which already exists in the result, as the input filter. Then a caller can list first and extract the chosen occurrences. ### References - `src/application/artifacts/ArtifactExtractionDestination.ts` - `docs/mcp-contracts.md` (Integrity record-and-continue / `extract_artifact`)

bsakaguchi commented

2026-10-16 21:03:57 +00:00

Author

Owner

@jsk1004ha commented on GitHub (Oct 15, 2026):

I'll work on this. I'll keep the default temporary destination, add an optional caller-selected output root, and select files by the existing inventory occurrence IDs. Extraction will still require a fresh destination and use the current containment, integrity, and cleanup checks. Nested archive members will remain outside this first change; callers can extract the containing archive and inspect it separately.

@jsk1004ha commented on GitHub (Oct 15, 2026): I'll work on this. I'll keep the default temporary destination, add an optional caller-selected output root, and select files by the existing inventory occurrence IDs. Extraction will still require a fresh destination and use the current containment, integrity, and cleanup checks. Nested archive members will remain outside this first change; callers can extract the containing archive and inspect it separately.

bsakaguchi commented

2026-10-16 21:03:58 +00:00

Author

Owner

@jsk1004ha commented on GitHub (Oct 15, 2026):

The implementation is on jsk1004ha:feat/artifact-selection-3490, pending the author's final review.

It adds --output-root and repeatable --occurrence-id flags, with output_root and occurrence_ids on MCP. Omitted options preserve the existing behavior. Unknown, duplicate, root and nested-member selections are rejected; existing destinations and destinations inside the source directory are refused. Full inventory checks remain separate from the selected files, so an intentional exclusion is not mistaken for missing source data.

Verification on commit 3d2920e2: 73 focused tests passed on Linux; 42 passed and one existing test skipped on macOS. The checks include real CLI/stdio MCP parity, source preservation, invalid-input refusals, rollback, and committed output after close. npm run check, test discovery, and npm run docs:check passed on Linux. Windows and the full repository test suite have not been run.

@jsk1004ha commented on GitHub (Oct 15, 2026): The implementation is on [jsk1004ha:feat/artifact-selection-3490](https://github.com/jsk1004ha/rea/compare/6fcfc5b5e0c063300cd5278dd8bad5b1ab1374f6...3d2920e2e4c9f31f1978015c8809b9071117a1b3), pending the author's final review. It adds `--output-root` and repeatable `--occurrence-id` flags, with `output_root` and `occurrence_ids` on MCP. Omitted options preserve the existing behavior. Unknown, duplicate, root and nested-member selections are rejected; existing destinations and destinations inside the source directory are refused. Full inventory checks remain separate from the selected files, so an intentional exclusion is not mistaken for missing source data. [Verification](https://github.com/jsk1004ha/rea/actions/runs/38702595598) on commit `3d2920e2`: 73 focused tests passed on Linux; 42 passed and one existing test skipped on macOS. The checks include real CLI/stdio MCP parity, source preservation, invalid-input refusals, rollback, and committed output after close. `npm run check`, test discovery, and `npm run docs:check` passed on Linux. Windows and the full repository test suite have not been run.

bsakaguchi referenced this issue

2026-10-17 21:10:11 +00:00

[PR #1302] [MERGED] fix(contracts): advertise NUL as the hex escape strict validators accept #2782

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#1302

No description provided.

Rows
Columns