morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[GH-ISSUE #3153] [Feature] Inventory Windows Installer (.msi / OLE CFB) packages read-only: streams, embedded cabinets and File-table names #1099

New issue

Open

opened 2026-10-15 18:31:03 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:31:03 +00:00

Owner

Originally created by @molako2 on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3153

Before submitting

I searched the open and closed issues and pull requests for a related proposal.
I understand the contribution scope must be reviewed before integration; open questions are welcome here.
This request does not include credentials, private samples, or sensitive target data.

Area

Evidence, artifacts, or reconstruction

Current limitation

REA cannot open a Windows Installer package. On 2026-10-14, I analysed a commercial Windows installer .msi. open_binary rejected it, so no REA operation could inventory the installed files. I worked around it outside REA with two steps:

7-Zip extraction of the OLE streams and embedded cabinet.
Read-only Windows Installer COM queries on the File, Component and Directory tables to map cabinet member keys to the real install layout.

That workaround needs a Windows host and the Windows Installer service. It produces no REA Evidence, and nothing records the provenance of the mapping.

Reproduction (synthetic, no vendor file)

Environment: Windows 11, Node.js 22.20.0, rea-agents 6.4.0 MCP server. The source was read at commit 583008852.

make_synthetic_msi.ps1 (under Verification details) builds a 20 480-byte MSI through the Windows Installer COM API. It contains one row each in Directory, Component, File and Media, plus a one-file MSZIP cabinet stored in _Streams as payload.cab (Media.Cabinet = "#payload.cab"). It installs and runs nothing.

open_binary { "path": "<abs>/synthetic.msi" }
→ { "code": "target_unavailable", "category": "unavailable",
    "message": "Cannot open artifact: unsupported binary format. ...",
    "details": { "reason": "unsupported binary format" } }

The same MSI nested inside a ZIP (setup/synthetic.msi) inventories as an opaque leaf. It has no limitation and no next probe:

{ "logical_path": "setup/synthetic.msi", "artifact_kind": "resource",
  "artifact_format": "file", "hash_status": "verified", "limitations": [] }

A minimal MS-CFB directory walk of the fixture (cfb_dir.py, under Verification details) shows the structure REA would need to expose. Raw names are packed into U+3800–U+4840. The decoded form is shown:

\x05SummaryInformation   180     [table]_StringPool   148
[table]_Tables             8     [table]_StringData   355
[table]_Columns          184     [table]Directory      12
[table]Component          12     [table]File           18
[table]Media              12     payload.cab          101   (MSCF cabinet)

Owning code

src/application/BinaryTargetResolver.ts:277 detectArtifactFormat recognises ZIP, binary or XML plist, .asar, .map, .js, xar! pkg and koly DMG. Any other input goes to executable-header parsing. parseExecutableHeader then returns err("unsupported binary format") at src/domain/binaryTarget.ts:88.
src/artifacts/inventory/ArtifactGraphConstruction.ts:451 classifyArtifactBytes has no CFB signature. A nested MSI therefore falls through to "file".
src/artifacts/inventory/reader.ts:22 createReader and inventoryCoverage choose readers. No CFB reader exists. For an unsupported container, a precedent exists: DMG and pkg roots on non-macOS hosts report coverage: { scope: "container-entries", status: "unknown" } plus a limitation.
src/domain/artifactGraph.ts:39 artifactFormatSchema has no msi or cfb member.

History: #363 ("Add MSIX/MSI/AppX package… analysis") was closed by PR #450. That PR added extension-only domain schemas (src/domain/packageAnalysis.ts) with no parser. Commit c93422a89 ("remove abandoned test-only analysis scaffolding") later deleted that module. MSI parsing never reached a public workflow.

Use cases

No response

Desired outcome and success criteria

Main question for maintainers: should REA inventory MSI/CFB as an archive (streams, embedded cabinets, File-table name mapping) without ever executing custom actions?

The following is a staged proposal. Each stage is useful alone.

Recognise and report (no new reader). Classify the CFB signature d0 cf 11 e0 a1 b1 1a e1 at the root and in nested occurrences. open_binary admits the file as an archive. inspect_artifact returns the root hash with coverage.status: "unknown" and a limitation such as "CFB child inventory reader is unavailable", the same pattern as DMG and pkg on non-macOS hosts. A nested MSI is no longer an unexplained "file".
Generic CFB reader. Streams and storages become occurrences. Each keeps its raw directory name, and also a decoded name when the MSI packing applies. The reader is pure TypeScript and follows MS-CFB v3/v4: header, DIFAT, FAT, miniFAT and directory red-black tree. Every sector chain is bounded and checked for cycles.
MSI interpretation. Parse _StringPool/_StringData, _Tables and _Columns, then File, Component, Directory and Media. Report the rows as observed facts. Report the rebuilt install path of each file as a derivation, never as an observation. DefaultDir and FileName carry short|long and target:source forms. Directory roots such as TARGETDIR and ProgramFilesFolder resolve only at install time, so property-dependent segments stay explicit unknowns.
Embedded cabinets. Expand _Streams cabinets that Media.Cabinet names with #. Expand MSZIP folders with the built-in zlib raw inflate, carrying the 32 KiB dictionary across CFDATA blocks. Report LZX and Quantum folders as unavailable unless maintainers approve a decoder. Report external cabinets (a Media.Cabinet value without #) as unresolved sibling references.

Success criteria:

open_binary on an MSI returns kind: "archive". It never returns target_unavailable.
inspect_artifact lists every stream, with coverage complete only when every stream was examined.
Cabinet members are occurrences whose identity is the cabinet member name, which is the File key. A separate, derived mapping gives File key → long name → directory path, with Evidence provenance.
CustomAction, InstallExecuteSequence, Binary and \x05DigitalSignature are inventoried as data only.
CLI and MCP produce the same contracts on Windows, Linux and macOS.

Expected interaction with targets

Read existing files or captured data

Target and scope details (optional)

Authority and side effects

Read-only bytes. No msiexec, no Windows Installer COM or msi.dll, no installer service, no registry access. COM also does not exist off Windows, which would break the cross-OS schema parity that #38 requires.
No execution or evaluation. Custom actions, conditions, sequences and formatted strings stay inert table data. Streams in the Binary table, such as custom-action DLLs or scripts, are occurrences. An analyst can open them with existing PE tooling, but REA never runs them.
Signatures. The \x05DigitalSignature stream is inventoried. Authenticode verification is out of scope for this issue.
Extraction. extract_artifact keeps the existing guarantees from SafeOutputTree: no traversal, no overwrite, verified cleanup. A design question remains open: should extraction use File keys (unique per package) or derived long names? Long names can collide, which links to #3152.
Resource bounds. Apply sector, stream-count, string-pool and decompression budgets (artifactDecompressionBudgetBytes) before allocation.

Open scope questions

Does MSI count as "ordinary archive and standard compression extraction" under CONTRIBUTING.md#contribution-scope? It is a relational database plus streams, not only an archive.
Should the reader inventory any CFB file generically, which also covers .msm, .msp, .mst and legacy Office files? Or should it apply only to MSI-shaped CFB files, recognised by _Tables and _StringPool?
Is the derived install layout part of the artifact graph, or a separate operation with its own Evidence? I propose that streams and cabinet members are graph occurrences and that the layout is a separate derived view.
LZX cabinets: add a dependency, write an in-house decoder, or report them as unavailable?

Verification available so far

I have a prototype or observed results

Verification details (optional)

Deterministic fixture writer. Add a TypeScript CFB/MSI writer under tests/fixtures/, in the style of src/dotnet/ManagedPe.fixture.ts. It emits string pool, table streams and an MSZIP cabinet, needs no host dependency, and adds no committed binaries.
Windows oracle test (optional, Windows CI only). Generate an MSI with make_synthetic_msi.ps1. Compare REA's File/Component/Directory projection with Windows Installer SELECT results.
Malformed inputs. FAT and DIFAT cycles, sector beyond EOF, a directory tree cycle, a miniFAT cut-off, a string-pool index out of range, a _Columns and table width mismatch, a CFDATA checksum mismatch, a truncated cabinet, an LZX folder, an external cabinet reference, and cancellation in the middle of a stream.
Workflows. CLI and MCP open_binary → inspect_artifact → extract_artifact. Also inspect an MSI nested in a ZIP.
Performance. Report tokens, elapsed time, CPU time and peak memory on one large public MSI, cold and warm. Do not commit that file.
Unverified. Real-world LZX cabinets, multi-volume media and patch packages (.msp) are not covered by the synthetic fixture.

Fixture generators (synthetic, no vendor file)

make_synthetic_msi.ps1

param([Parameter(Mandatory)] [string]$OutDir)
# Builds a synthetic Windows Installer database: Directory/Component/File/Media
# rows plus one embedded MSZIP cabinet in _Streams. Nothing is installed or run.
$ErrorActionPreference = 'Stop'
$msi = Join-Path $OutDir 'synthetic.msi'
$work = Join-Path $OutDir 'cab-src'
New-Item -ItemType Directory -Force $work | Out-Null
# The cabinet member name is the File table key, as Windows Installer requires.
[IO.File]::WriteAllText((Join-Path $work 'fil_chemin'), "synthetic payload`n")
& makecab.exe /D CompressionType=MSZIP (Join-Path $work 'fil_chemin') (Join-Path $OutDir 'payload.cab') | Out-Null

function Call($obj, $name, $kind, [object[]]$a) {
  # PowerShell wraps COM objects in PSObject; late-bound IDispatch needs the raw object.
  $raw = @($a | ForEach-Object { if ($_ -is [psobject]) { $_.psobject.BaseObject } else { $_ } })
  try { $obj.GetType().InvokeMember($name, $kind, $null, $obj, $raw) } catch { throw "$name failed: $_" }
}
$installer = New-Object -ComObject WindowsInstaller.Installer
if (Test-Path $msi) { Remove-Item $msi }
$db = Call $installer 'OpenDatabase' 'InvokeMethod' @($msi, 3)  # msiOpenDatabaseModeCreate
$sql = @(
  'CREATE TABLE `Directory` (`Directory` CHAR(72) NOT NULL, `Directory_Parent` CHAR(72), `DefaultDir` CHAR(255) NOT NULL LOCALIZABLE PRIMARY KEY `Directory`)',
  'CREATE TABLE `Component` (`Component` CHAR(72) NOT NULL, `ComponentId` CHAR(38), `Directory_` CHAR(72) NOT NULL, `Attributes` SHORT NOT NULL, `Condition` CHAR(255), `KeyPath` CHAR(72) PRIMARY KEY `Component`)',
  'CREATE TABLE `File` (`File` CHAR(72) NOT NULL, `Component_` CHAR(72) NOT NULL, `FileName` CHAR(255) NOT NULL LOCALIZABLE, `FileSize` LONG NOT NULL, `Version` CHAR(72), `Language` CHAR(20), `Attributes` SHORT, `Sequence` SHORT NOT NULL PRIMARY KEY `File`)',
  'CREATE TABLE `Media` (`DiskId` SHORT NOT NULL, `LastSequence` SHORT NOT NULL, `DiskPrompt` CHAR(64), `Cabinet` CHAR(255), `VolumeLabel` CHAR(32), `Source` CHAR(72) PRIMARY KEY `DiskId`)',
  "INSERT INTO ``Directory`` (``Directory``, ``Directory_Parent``, ``DefaultDir``) VALUES ('TARGETDIR', '', 'SourceDir')",
  "INSERT INTO ``Directory`` (``Directory``, ``Directory_Parent``, ``DefaultDir``) VALUES ('APPDIR', 'TARGETDIR', 'APPDIR~1|Application Synthetique')",
  "INSERT INTO ``Component`` (``Component``, ``ComponentId``, ``Directory_``, ``Attributes``, ``KeyPath``) VALUES ('cmp_chemin', '{00000000-0000-0000-0000-000000000001}', 'APPDIR', 0, 'fil_chemin')",
  "INSERT INTO ``File`` (``File``, ``Component_``, ``FileName``, ``FileSize``, ``Sequence``) VALUES ('fil_chemin', 'cmp_chemin', 'CHEMIN~1.TXT|Chemin long.txt', 18, 1)",
  "INSERT INTO ``Media`` (``DiskId``, ``LastSequence``, ``Cabinet``) VALUES (1, 1, '#payload.cab')"
)
foreach ($q in $sql) { $v = Call $db 'OpenView' 'InvokeMethod' @($q); Call $v 'Execute' 'InvokeMethod' @() | Out-Null; Call $v 'Close' 'InvokeMethod' @() | Out-Null }
$view = Call $db 'OpenView' 'InvokeMethod' @('SELECT `Name`, `Data` FROM `_Streams`')
Call $view 'Execute' 'InvokeMethod' @() | Out-Null
$rec = Call $installer 'CreateRecord' 'InvokeMethod' @(2)
Call $rec 'StringData' 'SetProperty' @(1, 'payload.cab') | Out-Null
Call $rec 'SetStream' 'InvokeMethod' @(2, (Join-Path $OutDir 'payload.cab')) | Out-Null
Call $view 'Modify' 'InvokeMethod' @(1, $rec) | Out-Null  # msiViewModifyInsert
Call $view 'Close' 'InvokeMethod' @() | Out-Null
$si = Call $db 'SummaryInformation' 'GetProperty' @(5)
Call $si 'Property' 'SetProperty' @(2, 'Synthetic REA fixture') | Out-Null
Call $si 'Property' 'SetProperty' @(7, 'Intel;1033') | Out-Null
Call $si 'Property' 'SetProperty' @(9, '{00000000-0000-0000-0000-0000000000AA}') | Out-Null
Call $si 'Persist' 'InvokeMethod' @() | Out-Null
Call $db 'Commit' 'InvokeMethod' @() | Out-Null
[System.Runtime.InteropServices.Marshal]::ReleaseComObject($db) | Out-Null
Get-Item $msi | Select-Object Name, Length

cfb_dir.py

# Minimal read-only CFB directory walk (MS-CFB 2.6) to show raw MSI stream names.
import sys, struct
data = open(sys.argv[1], "rb").read()
assert data[:8] == bytes.fromhex("d0cf11e0a1b11ae1")
sector = 1 << struct.unpack_from("<H", data, 30)[0]
first_dir = struct.unpack_from("<I", data, 48)[0]
difat = list(struct.unpack_from("<109I", data, 76))
fat = []
for s in difat:
    if s >= 0xFFFFFFFA: break
    fat += struct.unpack_from(f"<{sector//4}I", data, (s + 1) * sector)
s, entries = first_dir, []
while s < 0xFFFFFFFA:
    blk = data[(s + 1) * sector:(s + 2) * sector]
    for i in range(0, sector, 128):
        e = blk[i:i + 128]
        n = struct.unpack_from("<H", e, 64)[0]
        if n == 0: continue
        name = e[:n - 2].decode("utf-16le", "surrogatepass")
        kind = e[66]; size = struct.unpack_from("<Q", e, 120)[0]
        entries.append((kind, name, size))
    s = fat[s]
ALPHA = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._"
def msi_name(name):
    # Windows Installer packs stream names into U+3800..U+4840 (Wine/msitools decode_streamname).
    out = []
    for c in name:
        o = ord(c)
        if o == 0x4840: out.append("[table]")
        elif 0x3800 <= o < 0x4800: o -= 0x3800; out.append(ALPHA[o & 0x3F] + ALPHA[(o >> 6) & 0x3F])
        elif 0x4800 <= o < 0x4840: out.append(ALPHA[o - 0x4800])
        else: out.append(c)
    return "".join(out)
for kind, name, size in entries:
    print({1: "storage", 2: "stream", 5: "root"}.get(kind, kind), ascii(name), "->", ascii(msi_name(name)), size)

Alternatives considered

7-Zip plus Windows Installer COM (the 2026-10-14 workaround). It works only on Windows, produces no REA Evidence, and loses provenance between a cabinet member and its long name.
msitools, libmsi or wixl. These are external binaries. Under AGENTS.md, installers must not add unrelated software. These tools could serve as a test oracle only.
Extension-only classification (PR #450). It was removed as scaffolding because it inventoried nothing.

Possible direction

The plan under Desired outcome and success criteria is a proposal, not a requirement. No implementation is intended before a maintainer scope decision.

References

#38 (Windows artifact support), #363 and PR #450 (extension-only MSI schemas), commit c93422a89 (their removal)
#3152: duplicate ZIP paths; long-name collisions in MSI extraction would hit the same rule
[MS-CFB]: Compound File Binary File Format. Microsoft Cabinet Format (MS-CAB). Windows Installer database tables File, Component, Directory, Media

Originally created by @molako2 on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3153 ### Before submitting - [X] I searched the open and closed issues and pull requests for a related proposal. - [X] I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here. - [X] This request does not include credentials, private samples, or sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Current limitation REA cannot open a Windows Installer package. On 2026-10-14, I analysed a commercial Windows installer `.msi`. `open_binary` rejected it, so no REA operation could inventory the installed files. I worked around it outside REA with two steps: 1. 7-Zip extraction of the OLE streams and embedded cabinet. 2. Read-only Windows Installer COM queries on the `File`, `Component` and `Directory` tables to map cabinet member keys to the real install layout. That workaround needs a Windows host and the Windows Installer service. It produces no REA Evidence, and nothing records the provenance of the mapping. #### Reproduction (synthetic, no vendor file) Environment: Windows 11, Node.js 22.20.0, `rea-agents` 6.4.0 MCP server. The source was read at commit 583008852. `make_synthetic_msi.ps1` (under *Verification details*) builds a 20 480-byte MSI through the Windows Installer COM API. It contains one row each in `Directory`, `Component`, `File` and `Media`, plus a one-file MSZIP cabinet stored in `_Streams` as `payload.cab` (`Media.Cabinet = "#payload.cab"`). It installs and runs nothing. ```text open_binary { "path": "<abs>/synthetic.msi" } → { "code": "target_unavailable", "category": "unavailable", "message": "Cannot open artifact: unsupported binary format. ...", "details": { "reason": "unsupported binary format" } } ``` The same MSI nested inside a ZIP (`setup/synthetic.msi`) inventories as an opaque leaf. It has no limitation and no next probe: ```json { "logical_path": "setup/synthetic.msi", "artifact_kind": "resource", "artifact_format": "file", "hash_status": "verified", "limitations": [] } ``` A minimal MS-CFB directory walk of the fixture (`cfb_dir.py`, under *Verification details*) shows the structure REA would need to expose. Raw names are packed into U+3800–U+4840. The decoded form is shown: ```text \x05SummaryInformation 180 [table]_StringPool 148 [table]_Tables 8 [table]_StringData 355 [table]_Columns 184 [table]Directory 12 [table]Component 12 [table]File 18 [table]Media 12 payload.cab 101 (MSCF cabinet) ``` #### Owning code - [`src/application/BinaryTargetResolver.ts:277`](https://github.com/morluto/rea/blob/58300885207133422af235e0f2b10658cf958b03/src/application/BinaryTargetResolver.ts#L277) `detectArtifactFormat` recognises ZIP, binary or XML plist, `.asar`, `.map`, `.js`, `xar!` pkg and `koly` DMG. Any other input goes to executable-header parsing. `parseExecutableHeader` then returns `err("unsupported binary format")` at [`src/domain/binaryTarget.ts:88`](https://github.com/morluto/rea/blob/58300885207133422af235e0f2b10658cf958b03/src/domain/binaryTarget.ts#L88). - [`src/artifacts/inventory/ArtifactGraphConstruction.ts:451`](https://github.com/morluto/rea/blob/58300885207133422af235e0f2b10658cf958b03/src/artifacts/inventory/ArtifactGraphConstruction.ts#L451) `classifyArtifactBytes` has no CFB signature. A nested MSI therefore falls through to `"file"`. - [`src/artifacts/inventory/reader.ts:22`](https://github.com/morluto/rea/blob/58300885207133422af235e0f2b10658cf958b03/src/artifacts/inventory/reader.ts#L22) `createReader` and `inventoryCoverage` choose readers. No CFB reader exists. For an unsupported container, a precedent exists: DMG and pkg roots on non-macOS hosts report `coverage: { scope: "container-entries", status: "unknown" }` plus a limitation. - [`src/domain/artifactGraph.ts:39`](https://github.com/morluto/rea/blob/58300885207133422af235e0f2b10658cf958b03/src/domain/artifactGraph.ts#L39) `artifactFormatSchema` has no `msi` or `cfb` member. History: #363 ("Add MSIX/MSI/AppX package… analysis") was closed by PR #450. That PR added extension-only domain schemas ([`src/domain/packageAnalysis.ts`](https://github.com/morluto/rea/blob/58300885207133422af235e0f2b10658cf958b03/src/domain/packageAnalysis.ts)) with no parser. Commit c93422a89 ("remove abandoned test-only analysis scaffolding") later deleted that module. MSI parsing never reached a public workflow. ### Use cases _No response_ ### Desired outcome and success criteria Main question for maintainers: **should REA inventory MSI/CFB as an archive (streams, embedded cabinets, File-table name mapping) without ever executing custom actions?** The following is a staged proposal. Each stage is useful alone. 1. **Recognise and report (no new reader).** Classify the CFB signature `d0 cf 11 e0 a1 b1 1a e1` at the root and in nested occurrences. `open_binary` admits the file as an archive. `inspect_artifact` returns the root hash with `coverage.status: "unknown"` and a limitation such as "CFB child inventory reader is unavailable", the same pattern as DMG and pkg on non-macOS hosts. A nested MSI is no longer an unexplained `"file"`. 2. **Generic CFB reader.** Streams and storages become occurrences. Each keeps its raw directory name, and also a decoded name when the MSI packing applies. The reader is pure TypeScript and follows MS-CFB v3/v4: header, DIFAT, FAT, miniFAT and directory red-black tree. Every sector chain is bounded and checked for cycles. 3. **MSI interpretation.** Parse `_StringPool`/`_StringData`, `_Tables` and `_Columns`, then `File`, `Component`, `Directory` and `Media`. Report the rows as observed facts. Report the rebuilt install path of each file as a **derivation**, never as an observation. `DefaultDir` and `FileName` carry `short|long` and `target:source` forms. Directory roots such as `TARGETDIR` and `ProgramFilesFolder` resolve only at install time, so property-dependent segments stay explicit unknowns. 4. **Embedded cabinets.** Expand `_Streams` cabinets that `Media.Cabinet` names with `#`. Expand MSZIP folders with the built-in `zlib` raw inflate, carrying the 32 KiB dictionary across `CFDATA` blocks. Report LZX and Quantum folders as `unavailable` unless maintainers approve a decoder. Report external cabinets (a `Media.Cabinet` value without `#`) as unresolved sibling references. Success criteria: - `open_binary` on an MSI returns `kind: "archive"`. It never returns `target_unavailable`. - `inspect_artifact` lists every stream, with coverage `complete` only when every stream was examined. - Cabinet members are occurrences whose identity is the cabinet member name, which is the `File` key. A separate, derived mapping gives `File` key → long name → directory path, with Evidence provenance. - `CustomAction`, `InstallExecuteSequence`, `Binary` and `\x05DigitalSignature` are inventoried as data only. - CLI and MCP produce the same contracts on Windows, Linux and macOS. ### Expected interaction with targets Read existing files or captured data ### Target and scope details (optional) #### Authority and side effects - **Read-only bytes.** No `msiexec`, no Windows Installer COM or `msi.dll`, no installer service, no registry access. COM also does not exist off Windows, which would break the cross-OS schema parity that #38 requires. - **No execution or evaluation.** Custom actions, conditions, sequences and formatted strings stay inert table data. Streams in the `Binary` table, such as custom-action DLLs or scripts, are occurrences. An analyst can open them with existing PE tooling, but REA never runs them. - **Signatures.** The `\x05DigitalSignature` stream is inventoried. Authenticode verification is out of scope for this issue. - **Extraction.** `extract_artifact` keeps the existing guarantees from `SafeOutputTree`: no traversal, no overwrite, verified cleanup. A design question remains open: should extraction use `File` keys (unique per package) or derived long names? Long names can collide, which links to #3152. - **Resource bounds.** Apply sector, stream-count, string-pool and decompression budgets (`artifactDecompressionBudgetBytes`) before allocation. #### Open scope questions 1. Does MSI count as "ordinary archive and standard compression extraction" under `CONTRIBUTING.md#contribution-scope`? It is a relational database plus streams, not only an archive. 2. Should the reader inventory any CFB file generically, which also covers `.msm`, `.msp`, `.mst` and legacy Office files? Or should it apply only to MSI-shaped CFB files, recognised by `_Tables` and `_StringPool`? 3. Is the derived install layout part of the artifact graph, or a separate operation with its own Evidence? I propose that streams and cabinet members are graph occurrences and that the layout is a separate derived view. 4. LZX cabinets: add a dependency, write an in-house decoder, or report them as `unavailable`? ### Verification available so far I have a prototype or observed results ### Verification details (optional) - **Deterministic fixture writer.** Add a TypeScript CFB/MSI writer under `tests/fixtures/`, in the style of [`src/dotnet/ManagedPe.fixture.ts`](https://github.com/morluto/rea/blob/58300885207133422af235e0f2b10658cf958b03/src/dotnet/ManagedPe.fixture.ts). It emits string pool, table streams and an MSZIP cabinet, needs no host dependency, and adds no committed binaries. - **Windows oracle test (optional, Windows CI only).** Generate an MSI with `make_synthetic_msi.ps1`. Compare REA's `File`/`Component`/`Directory` projection with Windows Installer `SELECT` results. - **Malformed inputs.** FAT and DIFAT cycles, sector beyond EOF, a directory tree cycle, a miniFAT cut-off, a string-pool index out of range, a `_Columns` and table width mismatch, a `CFDATA` checksum mismatch, a truncated cabinet, an LZX folder, an external cabinet reference, and cancellation in the middle of a stream. - **Workflows.** CLI and MCP `open_binary` → `inspect_artifact` → `extract_artifact`. Also inspect an MSI nested in a ZIP. - **Performance.** Report tokens, elapsed time, CPU time and peak memory on one large public MSI, cold and warm. Do not commit that file. - **Unverified.** Real-world LZX cabinets, multi-volume media and patch packages (`.msp`) are not covered by the synthetic fixture. #### Fixture generators (synthetic, no vendor file) <details><summary><code>make_synthetic_msi.ps1</code></summary> ```powershell param([Parameter(Mandatory)] [string]$OutDir) # Builds a synthetic Windows Installer database: Directory/Component/File/Media # rows plus one embedded MSZIP cabinet in _Streams. Nothing is installed or run. $ErrorActionPreference = 'Stop' $msi = Join-Path $OutDir 'synthetic.msi' $work = Join-Path $OutDir 'cab-src' New-Item -ItemType Directory -Force $work | Out-Null # The cabinet member name is the File table key, as Windows Installer requires. [IO.File]::WriteAllText((Join-Path $work 'fil_chemin'), "synthetic payload`n") & makecab.exe /D CompressionType=MSZIP (Join-Path $work 'fil_chemin') (Join-Path $OutDir 'payload.cab') | Out-Null function Call($obj, $name, $kind, [object[]]$a) { # PowerShell wraps COM objects in PSObject; late-bound IDispatch needs the raw object. $raw = @($a | ForEach-Object { if ($_ -is [psobject]) { $_.psobject.BaseObject } else { $_ } }) try { $obj.GetType().InvokeMember($name, $kind, $null, $obj, $raw) } catch { throw "$name failed: $_" } } $installer = New-Object -ComObject WindowsInstaller.Installer if (Test-Path $msi) { Remove-Item $msi } $db = Call $installer 'OpenDatabase' 'InvokeMethod' @($msi, 3) # msiOpenDatabaseModeCreate $sql = @( 'CREATE TABLE `Directory` (`Directory` CHAR(72) NOT NULL, `Directory_Parent` CHAR(72), `DefaultDir` CHAR(255) NOT NULL LOCALIZABLE PRIMARY KEY `Directory`)', 'CREATE TABLE `Component` (`Component` CHAR(72) NOT NULL, `ComponentId` CHAR(38), `Directory_` CHAR(72) NOT NULL, `Attributes` SHORT NOT NULL, `Condition` CHAR(255), `KeyPath` CHAR(72) PRIMARY KEY `Component`)', 'CREATE TABLE `File` (`File` CHAR(72) NOT NULL, `Component_` CHAR(72) NOT NULL, `FileName` CHAR(255) NOT NULL LOCALIZABLE, `FileSize` LONG NOT NULL, `Version` CHAR(72), `Language` CHAR(20), `Attributes` SHORT, `Sequence` SHORT NOT NULL PRIMARY KEY `File`)', 'CREATE TABLE `Media` (`DiskId` SHORT NOT NULL, `LastSequence` SHORT NOT NULL, `DiskPrompt` CHAR(64), `Cabinet` CHAR(255), `VolumeLabel` CHAR(32), `Source` CHAR(72) PRIMARY KEY `DiskId`)', "INSERT INTO ``Directory`` (``Directory``, ``Directory_Parent``, ``DefaultDir``) VALUES ('TARGETDIR', '', 'SourceDir')", "INSERT INTO ``Directory`` (``Directory``, ``Directory_Parent``, ``DefaultDir``) VALUES ('APPDIR', 'TARGETDIR', 'APPDIR~1|Application Synthetique')", "INSERT INTO ``Component`` (``Component``, ``ComponentId``, ``Directory_``, ``Attributes``, ``KeyPath``) VALUES ('cmp_chemin', '{00000000-0000-0000-0000-000000000001}', 'APPDIR', 0, 'fil_chemin')", "INSERT INTO ``File`` (``File``, ``Component_``, ``FileName``, ``FileSize``, ``Sequence``) VALUES ('fil_chemin', 'cmp_chemin', 'CHEMIN~1.TXT|Chemin long.txt', 18, 1)", "INSERT INTO ``Media`` (``DiskId``, ``LastSequence``, ``Cabinet``) VALUES (1, 1, '#payload.cab')" ) foreach ($q in $sql) { $v = Call $db 'OpenView' 'InvokeMethod' @($q); Call $v 'Execute' 'InvokeMethod' @() | Out-Null; Call $v 'Close' 'InvokeMethod' @() | Out-Null } $view = Call $db 'OpenView' 'InvokeMethod' @('SELECT `Name`, `Data` FROM `_Streams`') Call $view 'Execute' 'InvokeMethod' @() | Out-Null $rec = Call $installer 'CreateRecord' 'InvokeMethod' @(2) Call $rec 'StringData' 'SetProperty' @(1, 'payload.cab') | Out-Null Call $rec 'SetStream' 'InvokeMethod' @(2, (Join-Path $OutDir 'payload.cab')) | Out-Null Call $view 'Modify' 'InvokeMethod' @(1, $rec) | Out-Null # msiViewModifyInsert Call $view 'Close' 'InvokeMethod' @() | Out-Null $si = Call $db 'SummaryInformation' 'GetProperty' @(5) Call $si 'Property' 'SetProperty' @(2, 'Synthetic REA fixture') | Out-Null Call $si 'Property' 'SetProperty' @(7, 'Intel;1033') | Out-Null Call $si 'Property' 'SetProperty' @(9, '{00000000-0000-0000-0000-0000000000AA}') | Out-Null Call $si 'Persist' 'InvokeMethod' @() | Out-Null Call $db 'Commit' 'InvokeMethod' @() | Out-Null [System.Runtime.InteropServices.Marshal]::ReleaseComObject($db) | Out-Null Get-Item $msi | Select-Object Name, Length ``` </details> <details><summary><code>cfb_dir.py</code></summary> ```python # Minimal read-only CFB directory walk (MS-CFB 2.6) to show raw MSI stream names. import sys, struct data = open(sys.argv[1], "rb").read() assert data[:8] == bytes.fromhex("d0cf11e0a1b11ae1") sector = 1 << struct.unpack_from("<H", data, 30)[0] first_dir = struct.unpack_from("<I", data, 48)[0] difat = list(struct.unpack_from("<109I", data, 76)) fat = [] for s in difat: if s >= 0xFFFFFFFA: break fat += struct.unpack_from(f"<{sector//4}I", data, (s + 1) * sector) s, entries = first_dir, [] while s < 0xFFFFFFFA: blk = data[(s + 1) * sector:(s + 2) * sector] for i in range(0, sector, 128): e = blk[i:i + 128] n = struct.unpack_from("<H", e, 64)[0] if n == 0: continue name = e[:n - 2].decode("utf-16le", "surrogatepass") kind = e[66]; size = struct.unpack_from("<Q", e, 120)[0] entries.append((kind, name, size)) s = fat[s] ALPHA = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz._" def msi_name(name): # Windows Installer packs stream names into U+3800..U+4840 (Wine/msitools decode_streamname). out = [] for c in name: o = ord(c) if o == 0x4840: out.append("[table]") elif 0x3800 <= o < 0x4800: o -= 0x3800; out.append(ALPHA[o & 0x3F] + ALPHA[(o >> 6) & 0x3F]) elif 0x4800 <= o < 0x4840: out.append(ALPHA[o - 0x4800]) else: out.append(c) return "".join(out) for kind, name, size in entries: print({1: "storage", 2: "stream", 5: "root"}.get(kind, kind), ascii(name), "->", ascii(msi_name(name)), size) ``` </details> ### Alternatives considered - **7-Zip plus Windows Installer COM** (the 2026-10-14 workaround). It works only on Windows, produces no REA Evidence, and loses provenance between a cabinet member and its long name. - **`msitools`, `libmsi` or `wixl`.** These are external binaries. Under `AGENTS.md`, installers must not add unrelated software. These tools could serve as a test oracle only. - **Extension-only classification** (PR #450). It was removed as scaffolding because it inventoried nothing. ### Possible direction The plan under *Desired outcome and success criteria* is a proposal, not a requirement. No implementation is intended before a maintainer scope decision. ### References - #38 (Windows artifact support), #363 and PR #450 (extension-only MSI schemas), commit c93422a89 (their removal) - #3152: duplicate ZIP paths; long-name collisions in MSI extraction would hit the same rule - [MS-CFB]: Compound File Binary File Format. Microsoft Cabinet Format (MS-CAB). Windows Installer database tables `File`, `Component`, `Directory`, `Media`

bsakaguchi added the

enhancement

label

2026-10-15 18:31:03 +00:00

bsakaguchi referenced this issue

2026-10-17 21:08:36 +00:00

[PR #1099] [MERGED] fix(native): bind signature observations to registered target versions #2647

bsakaguchi referenced this issue

2026-10-17 21:08:42 +00:00

[PR #1107] [MERGED] ci: run signature target binding regressions on native macOS #2655

bsakaguchi added

enhancement