[GH-ISSUE #6463] [Docs] Unblob submodule checkout fails on Windows with `invalid path` #3875

New issue

Closed

opened 2026-10-31 00:44:50 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:44:50 +00:00

Owner

Originally created by @ezrabrilliant on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6463

Before submitting

I searched the existing issues for a related documentation report.
I removed credentials, tokens, private paths, and sensitive target data.

Documentation location

docs/firmware-analysis.md#caller-supplied-tools: the git submodule update --init third_party/binwalk third_party/unblob command. The same failure affects git submodule update --init --recursive from a Windows source checkout (CONTRIBUTING.md#windows-source-checkouts does not mention it).

Problem

On Windows, the documented command cannot check out the pinned Unblob submodule (1fcc7a0a, 26.6.4) and exits 128:

error: invalid path 'tests/integration/archive/ar/__output__/a1x01.ar_extract/�?rvíztűrő tükörfúrógép'
fatal: Unable to checkout '1fcc7a0a584a70a96c31f5a276c20944d199a089' in submodule path 'third_party/unblob'

At that commit, 11 upstream fixture paths under tests/integration/ cannot exist on NTFS. They contain ?, ", a carriage return or a control byte, or end with dots or a space. For example:

tests/integration/archive/tar/__output__/dual_entry.tar_extract/ppc64-POWER7-64cpu/proc/device-tree/hmc-managed?
tests/integration/filesystem/extfs/__output__/debugfs.quoting.img_extract/evil"dir/nested.ext
tests/integration/archive/dmg/__output__/fruits_osx.dmg_extract/fruits/.HFS+ Private Directory Data\r/.gitkeep
tests/integration/compression/gzip/__output__/path-traversal.gz_extract/whatever......

This affects Windows users who fetch the firmware sources and Windows contributors who run git submodule update --init --recursive. With --jobs 5, that command also left third_party/wakaru cloned but not checked out at its pinned commit. That second effect probably depends on the order in which submodules are processed.

Expected content

Proposed: a Windows note with a verified workaround. REA does not use Unblob's own test fixtures, so the guide can exclude them:

git -C third_party/unblob config core.protectNTFS false
git -C third_party/unblob sparse-checkout set --no-cone '/*' '!/tests/integration/'
git submodule update third_party/unblob

The note should also say that both settings are required, that the setting applies only to the Unblob submodule, and that Git Bash users must set MSYS_NO_PATHCONV=1 for the sparse-checkout command. CONTRIBUTING's Windows section can link to it for the --recursive case.

This concerns only the source checkout. Firmware analysis itself remains documented as verified on Linux x64.

Supporting evidence

Observed on Windows 11 Pro 10.0.26200 x64, Git 2.47.1.windows.1, git-lfs 3.6.0, REA source e32a0494d. Before each test, the Unblob submodule was reset with git submodule deinit -f and its .git/modules directory was removed.

Test	Result
Documented command, unchanged	exit 128, `invalid path`
Sparse pattern `'/*' '!/tests/integration/'` only, with `core.protectNTFS` at its default	retry exit 1, 11 `invalid path` errors, one per affected path
`core.protectNTFS false` with a pattern that did not exclude `tests/integration/`	exit 128, `cannot create directory at 'tests/integration/.../.HFS+ Private Directory Data?': Invalid argument`
Both settings, then `git submodule update third_party/unblob`	exit 0; `1fcc7a0a… third_party/unblob (26.6.4)`; 249 files checked out, 2837 skip-worktree, clean submodule and superproject status
Same steps, with the sparse pattern applied first	same successful result
`git submodule update --init --recursive` after the workaround	exit 0; all five submodules at their pinned commits

Git Bash rewrites the exclusion pattern. Output of git rev-parse --sq-quote '/*' '!/tests/integration/':

plain Git Bash:      '/*' ''\!'C:/Program Files/Git/tests/integration/'
MSYS_NO_PATHCONV=1:  '/*' ''\!'/tests/integration/'

Not tested: other Git for Windows versions, and checkouts made inside WSL.

Additional context

I have a docs PR ready that adds a "Windows submodule checkout" subsection to docs/firmware-analysis.md and a pointer from CONTRIBUTING.md#windows-source-checkouts. npm run docs:check passes locally.

Originally created by @ezrabrilliant on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6463 ### Before submitting - [x] I searched the existing issues for a related documentation report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Documentation location [docs/firmware-analysis.md#caller-supplied-tools](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md#caller-supplied-tools): the `git submodule update --init third_party/binwalk third_party/unblob` command. The same failure affects `git submodule update --init --recursive` from a Windows source checkout ([CONTRIBUTING.md#windows-source-checkouts](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#windows-source-checkouts) does not mention it). ### Problem On Windows, the documented command cannot check out the pinned Unblob submodule (`1fcc7a0a`, 26.6.4) and exits 128: ```text error: invalid path 'tests/integration/archive/ar/__output__/a1x01.ar_extract/�?rvíztűrő tükörfúrógép' fatal: Unable to checkout '1fcc7a0a584a70a96c31f5a276c20944d199a089' in submodule path 'third_party/unblob' ``` At that commit, 11 upstream fixture paths under `tests/integration/` cannot exist on NTFS. They contain `?`, `"`, a carriage return or a control byte, or end with dots or a space. For example: ```text tests/integration/archive/tar/__output__/dual_entry.tar_extract/ppc64-POWER7-64cpu/proc/device-tree/hmc-managed? tests/integration/filesystem/extfs/__output__/debugfs.quoting.img_extract/evil"dir/nested.ext tests/integration/archive/dmg/__output__/fruits_osx.dmg_extract/fruits/.HFS+ Private Directory Data\r/.gitkeep tests/integration/compression/gzip/__output__/path-traversal.gz_extract/whatever...... ``` This affects Windows users who fetch the firmware sources and Windows contributors who run `git submodule update --init --recursive`. With `--jobs 5`, that command also left `third_party/wakaru` cloned but not checked out at its pinned commit. That second effect probably depends on the order in which submodules are processed. ### Expected content Proposed: a Windows note with a verified workaround. REA does not use Unblob's own test fixtures, so the guide can exclude them: ```powershell git -C third_party/unblob config core.protectNTFS false git -C third_party/unblob sparse-checkout set --no-cone '/*' '!/tests/integration/' git submodule update third_party/unblob ``` The note should also say that both settings are required, that the setting applies only to the Unblob submodule, and that Git Bash users must set `MSYS_NO_PATHCONV=1` for the `sparse-checkout` command. CONTRIBUTING's Windows section can link to it for the `--recursive` case. This concerns only the source checkout. Firmware analysis itself remains documented as verified on Linux x64. ### Supporting evidence Observed on Windows 11 Pro 10.0.26200 x64, Git 2.47.1.windows.1, git-lfs 3.6.0, REA source `e32a0494d`. Before each test, the Unblob submodule was reset with `git submodule deinit -f` and its `.git/modules` directory was removed. | Test | Result | | --- | --- | | Documented command, unchanged | exit 128, `invalid path` | | Sparse pattern `'/*' '!/tests/integration/'` only, with `core.protectNTFS` at its default | retry exit 1, 11 `invalid path` errors, one per affected path | | `core.protectNTFS false` with a pattern that did not exclude `tests/integration/` | exit 128, `cannot create directory at 'tests/integration/.../.HFS+ Private Directory Data?': Invalid argument` | | Both settings, then `git submodule update third_party/unblob` | exit 0; ` 1fcc7a0a… third_party/unblob (26.6.4)`; 249 files checked out, 2837 skip-worktree, clean submodule and superproject status | | Same steps, with the sparse pattern applied first | same successful result | | `git submodule update --init --recursive` after the workaround | exit 0; all five submodules at their pinned commits | Git Bash rewrites the exclusion pattern. Output of `git rev-parse --sq-quote '/*' '!/tests/integration/'`: ```text plain Git Bash: '/*' ''\!'C:/Program Files/Git/tests/integration/' MSYS_NO_PATHCONV=1: '/*' ''\!'/tests/integration/' ``` Not tested: other Git for Windows versions, and checkouts made inside WSL. ### Additional context I have a docs PR ready that adds a "Windows submodule checkout" subsection to `docs/firmware-analysis.md` and a pointer from `CONTRIBUTING.md#windows-source-checkouts`. `npm run docs:check` passes locally.

bsakaguchi closed this issue

2026-10-31 00:44:50 +00:00

bsakaguchi referenced this issue

2026-10-31 00:47:21 +00:00

[GH-ISSUE #6818] [Bug] Mach-O section and segment names lose surrounding whitespace #4166

bsakaguchi referenced this issue

2026-10-31 01:11:14 +00:00

[PR #4051] [MERGED] fix(native): report an object file's unnamed segment by its empty name #6021

bsakaguchi referenced this issue

2026-11-01 21:47:46 +00:00

[PR #6842] [MERGED] fix(native): preserve whitespace in Mach-O segment and section names #7424