[PR #6464] [MERGED] docs(firmware): document the Windows Unblob submodule checkout #7358

Closed
opened 2026-11-01 21:47:18 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/6464
Author: @ezrabrilliant
Created: 10/23/2026
Status: ✅ Merged
Merged: 10/23/2026
Merged by: @morluto

Base: main ← Head: docs/windows-unblob-submodule


📝 Commits (1)

  • 624be5a docs(firmware): document the Windows Unblob submodule checkout

📊 Changes

2 files changed (+26 additions, -0 deletions)

View changed files

📝 CONTRIBUTING.md (+5 -0)
📝 docs/firmware-analysis.md (+21 -0)

📄 Description

Summary

Fixes #6463.

On Windows, git submodule update --init third_party/binwalk third_party/unblob from docs/firmware-analysis.md cannot check out the pinned Unblob submodule. git submodule update --init --recursive fails the same way. This PR documents a verified workaround and points Windows contributors to it.

Problem and expected behavior

The pinned Unblob commit (1fcc7a0a, 26.6.4) contains 11 upstream fixture paths under tests/integration/ that are invalid on NTFS. They contain ?, ", a carriage return or a control byte, or end with dots or a space. The checkout exits 128 with error: invalid path. REA does not use those fixtures, so excluding them yields a usable source checkout.

Change and scope

  • docs/firmware-analysis.md: a new "Windows submodule checkout" subsection. It gives three PowerShell commands (core.protectNTFS false for the Unblob submodule only, a non-cone sparse pattern that excludes /tests/integration/, and a retry), explains why both settings are required, and covers Git Bash path rewriting.
  • CONTRIBUTING.md#windows-source-checkouts: one paragraph linking to that subsection for --recursive checkouts.

Not included: changes to upstream Unblob sources (they stay unmodified), the gitlink, or firmware analysis support on Windows, which remains documented as verified on Linux x64.

Contribution scope: documentation within existing boundaries; no maintainer scope decision required.

Contract and boundary impact

  • Semantic owner and earliest changed stage: documentation only.

  • Contribution scope fit and linked maintainer decision: not applicable (documentation fix).

  • CLI and MCP/tool-catalog contract: none.

  • Provider, bridge, target-format, or platform compatibility: documents a Windows source-checkout workaround; no runtime change.

  • Evidence, artifact, provenance, or reconstruction contract: none.

  • Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable.

  • Generated metadata, package, or installation impact: none; docs/ and CONTRIBUTING.md only.

  • Read existing files or captured data.

  • Launch or attach to a process.

  • Inject instrumentation or run scripts/code.

  • Change process memory, registers, files, or configuration.

  • Contact network services.

  • Recover protected code or work around protections.

  • Documentation or usability only; no target interaction.

  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Executed on Windows 11 Pro 10.0.26200 x64, Git 2.47.1.windows.1, git-lfs 3.6.0, base e32a0494d. Before each run, the Unblob submodule was reset with git submodule deinit -f and its .git/modules directory was removed.

  • Tests added or updated: none (documentation only).
  • Base reproduction or other evidence:
    • The documented command, unchanged, exits 128 with invalid path.
    • The sparse pattern alone, with core.protectNTFS at its default, makes the retry exit 1 with 11 invalid path errors.
    • core.protectNTFS false without an effective exclusion fails with cannot create directory ... Invalid argument.
    • With both settings, the retry exits 0 and git submodule status shows 1fcc7a0a… third_party/unblob (26.6.4). The submodule has 249 files checked out and 2837 skip-worktree entries, and both the submodule and the superproject status are clean.
    • Applying the sparse pattern before or after the config change gives the same result.
    • After the workaround, git submodule update --init --recursive exits 0 with all five submodules at their pinned commits.
    • In Git Bash, git rev-parse --sq-quote '!/tests/integration/' shows the pattern rewritten to !C:/Program Files/Git/tests/integration/. With MSYS_NO_PATHCONV=1 it stays !/tests/integration/.
  • User-visible CLI/MCP output (if applicable): not applicable.
  • Remaining proof gaps: other Git for Windows versions; checkouts made inside WSL.

Validation performed

  • npm run docs:check: passed (6/6 Turbo tasks; readmeConsistency.test.ts 9/9). This ran on Windows with native/windows/build/ restored from the rea-agents@6.7.0 tarball, as the run-exclusive error message instructs. Those files are ignored and not committed.
  • npm run docs:build: passed. The rendered firmware-analysis.html contains id="windows-submodule-checkout", the anchor that the CONTRIBUTING link targets.
  • Pre-push check:fast (typecheck, lint): passed, 2/2 Turbo tasks.
  • The workaround sequence above was run end-to-end from a fresh submodule state, with the results listed.

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: Windows 11 x64 (Git checkout only); no provider run.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: the workaround disables core.protectNTFS only in the Unblob submodule's own Git config. The sparse pattern keeps the NTFS-invalid fixture names from being written to disk.

Author readiness

Human owner: @ezrabrilliant

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LjhoPGpqAwHwcCgectzRsp


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/6464 **Author:** [@ezrabrilliant](https://github.com/ezrabrilliant) **Created:** 10/23/2026 **Status:** ✅ Merged **Merged:** 10/23/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `docs/windows-unblob-submodule` --- ### 📝 Commits (1) - [`624be5a`](https://github.com/morluto/rea/commit/624be5af56c14988c76c4ad7b8abbb1af453eb45) docs(firmware): document the Windows Unblob submodule checkout ### 📊 Changes **2 files changed** (+26 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `CONTRIBUTING.md` (+5 -0) 📝 `docs/firmware-analysis.md` (+21 -0) </details> ### 📄 Description ## Summary Fixes #6463. On Windows, `git submodule update --init third_party/binwalk third_party/unblob` from [docs/firmware-analysis.md](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md#caller-supplied-tools) cannot check out the pinned Unblob submodule. `git submodule update --init --recursive` fails the same way. This PR documents a verified workaround and points Windows contributors to it. ## Problem and expected behavior The pinned Unblob commit (`1fcc7a0a`, 26.6.4) contains 11 upstream fixture paths under `tests/integration/` that are invalid on NTFS. They contain `?`, `"`, a carriage return or a control byte, or end with dots or a space. The checkout exits 128 with `error: invalid path`. REA does not use those fixtures, so excluding them yields a usable source checkout. ## Change and scope - `docs/firmware-analysis.md`: a new "Windows submodule checkout" subsection. It gives three PowerShell commands (`core.protectNTFS false` for the Unblob submodule only, a non-cone sparse pattern that excludes `/tests/integration/`, and a retry), explains why both settings are required, and covers Git Bash path rewriting. - `CONTRIBUTING.md#windows-source-checkouts`: one paragraph linking to that subsection for `--recursive` checkouts. Not included: changes to upstream Unblob sources (they stay unmodified), the gitlink, or firmware analysis support on Windows, which remains documented as verified on Linux x64. Contribution scope: documentation within existing boundaries; no maintainer scope decision required. ## Contract and boundary impact - Semantic owner and earliest changed stage: documentation only. - Contribution scope fit and linked maintainer decision: not applicable (documentation fix). - CLI and MCP/tool-catalog contract: none. - Provider, bridge, target-format, or platform compatibility: documents a Windows source-checkout workaround; no runtime change. - Evidence, artifact, provenance, or reconstruction contract: none. - Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable. - Generated metadata, package, or installation impact: none; `docs/` and `CONTRIBUTING.md` only. - [ ] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [x] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage Executed on Windows 11 Pro 10.0.26200 x64, Git 2.47.1.windows.1, git-lfs 3.6.0, base `e32a0494d`. Before each run, the Unblob submodule was reset with `git submodule deinit -f` and its `.git/modules` directory was removed. - Tests added or updated: none (documentation only). - Base reproduction or other evidence: - The documented command, unchanged, exits 128 with `invalid path`. - The sparse pattern alone, with `core.protectNTFS` at its default, makes the retry exit 1 with 11 `invalid path` errors. - `core.protectNTFS false` without an effective exclusion fails with `cannot create directory ... Invalid argument`. - With both settings, the retry exits 0 and `git submodule status` shows ` 1fcc7a0a… third_party/unblob (26.6.4)`. The submodule has 249 files checked out and 2837 skip-worktree entries, and both the submodule and the superproject status are clean. - Applying the sparse pattern before or after the config change gives the same result. - After the workaround, `git submodule update --init --recursive` exits 0 with all five submodules at their pinned commits. - In Git Bash, `git rev-parse --sq-quote '!/tests/integration/'` shows the pattern rewritten to `!C:/Program Files/Git/tests/integration/`. With `MSYS_NO_PATHCONV=1` it stays `!/tests/integration/`. - User-visible CLI/MCP output (if applicable): not applicable. - Remaining proof gaps: other Git for Windows versions; checkouts made inside WSL. ## Validation performed - `npm run docs:check`: passed (6/6 Turbo tasks; `readmeConsistency.test.ts` 9/9). This ran on Windows with `native/windows/build/` restored from the `rea-agents@6.7.0` tarball, as the `run-exclusive` error message instructs. Those files are ignored and not committed. - `npm run docs:build`: passed. The rendered `firmware-analysis.html` contains `id="windows-submodule-checkout"`, the anchor that the CONTRIBUTING link targets. - Pre-push `check:fast` (`typecheck`, `lint`): passed, 2/2 Turbo tasks. - The workaround sequence above was run end-to-end from a fresh submodule state, with the results listed. ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: Windows 11 x64 (Git checkout only); no provider run. - Package or release metadata impact: none. - Security, privacy, process, or containment review: the workaround disables `core.protectNTFS` only in the Unblob submodule's own Git config. The sparse pattern keeps the NTFS-invalid fixture names from being written to disk. ## Author readiness Human owner: @ezrabrilliant - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [ ] This PR addresses a concrete problem or an agreed enhancement. - [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [ ] The PR has one focused outcome and the title follows `type(scope): outcome`. - [ ] Related issue is linked, or the reason for not linking one is stated above. - [ ] Tests cover changed observable behavior and meaningful failure paths. - [ ] Owning docs, contracts, and generated metadata are updated where needed. - [ ] User-visible CLI/MCP changes include representative output. - [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LjhoPGpqAwHwcCgectzRsp --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-11-01 21:47:18 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#7358
No description provided.