[GH-ISSUE #3237] Legacy ZIP filenames are guessed as UTF-8 and distinct members collide #1161

New issue

Closed

opened 2026-10-15 18:31:37 +00:00 by bsakaguchi · 1 comment

bsakaguchi commented

2026-10-15 18:31:37 +00:00

Owner

Originally created by @xu91102 on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3237

A valid legacy ZIP filename is silently reinterpreted as UTF-8 when its bytes also form a valid UTF-8 sequence. With both encodings present, inspect-artifact rejects two distinct members as a path collision.

Reproduced on main 500e32399a3bc827e17fb4adaa97db3bc0df29b5, Node 24.18.0, macOS arm64, locked dependencies and compiled public CLI. Python zipfile independently reads both members and verifies CRCs.

from pathlib import Path
import base64, zipfile
archive = Path("names.zip")
archive.write_bytes(base64.b64decode("UEsDBBQAAAAAAAAAIQDuCcsXBgAAAAYAAAAGAAAAw6kudHh0bGVnYWN5UEsDBBQAAAgAAAAAIQCBO/g1BAAAAAQAAAAGAAAAw6kudHh0dXRmOFBLAQIUABQAAAAAAAAAIQDuCcsXBgAAAAYAAAAGAAAAAAAAAAAAAAAAAAAAAADDqS50eHRQSwECFAAUAAAIAAAAACEAgTv4NQQAAAAEAAAABgAAAAAAAAAAAAAAAAAqAAAAw6kudHh0UEsFBgAAAAACAAIAaAAAAFIAAAAAAA=="))
with zipfile.ZipFile(archive) as z:
    print([(i.filename, i.flag_bits, z.read(i)) for i in z.infolist()])
    assert z.testzip() is None

Expected names/content: ├⌐.txt, flag 0, legacy; and é.txt, flag 2048, utf8. Both entries contain the same raw filename bytes c3 a9 2e 74 78 74 but have different general-purpose bit 11 settings. There is no Unicode Path extra field.

Run the built public workflow:

node scripts/rea.mjs inspect-artifact "$PWD/names.zip" --json

Actual exit 1, artifact_operation_failed, Artifact path collision: é.txt. With only the first member, the command exits 0 but incorrectly inventories é.txt instead of ├⌐.txt. ASCII, CP437 byte 0x82 and explicitly UTF-8 flagged controls all succeed.

ZipArtifactReader supplies REA's CP437 decoder but leaves filenameEncoding unspecified. The installed zip.js reader then guesses UTF-8 before invoking that decoder. The name owner needs an explicit legacy fallback while retaining bit-11 UTF-8 and valid Unicode Path overrides. Path/collision, CRC, overlap and size validation should stay intact.

This is within the existing ZIP support and the declared behavior of merged #2365, whose distinct change handled ASCII control characters. That PR does not disable the encoding guess. PKWARE APPNOTE Appendix D.1–D.2 specifies the legacy character set when bit 11 is unset and UTF-8 when it is set. Current targeted issue/PR searches for CP437, filenameEncoding and UTF8 flag found no open fix for this case.

Originally created by @xu91102 on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3237 A valid legacy ZIP filename is silently reinterpreted as UTF-8 when its bytes also form a valid UTF-8 sequence. With both encodings present, `inspect-artifact` rejects two distinct members as a path collision. Reproduced on main `500e32399a3bc827e17fb4adaa97db3bc0df29b5`, Node 24.18.0, macOS arm64, locked dependencies and compiled public CLI. Python `zipfile` independently reads both members and verifies CRCs. ```python from pathlib import Path import base64, zipfile archive = Path("names.zip") archive.write_bytes(base64.b64decode("UEsDBBQAAAAAAAAAIQDuCcsXBgAAAAYAAAAGAAAAw6kudHh0bGVnYWN5UEsDBBQAAAgAAAAAIQCBO/g1BAAAAAQAAAAGAAAAw6kudHh0dXRmOFBLAQIUABQAAAAAAAAAIQDuCcsXBgAAAAYAAAAGAAAAAAAAAAAAAAAAAAAAAADDqS50eHRQSwECFAAUAAAIAAAAACEAgTv4NQQAAAAEAAAABgAAAAAAAAAAAAAAAAAqAAAAw6kudHh0UEsFBgAAAAACAAIAaAAAAFIAAAAAAA==")) with zipfile.ZipFile(archive) as z: print([(i.filename, i.flag_bits, z.read(i)) for i in z.infolist()]) assert z.testzip() is None ``` Expected names/content: `├⌐.txt`, flag 0, `legacy`; and `é.txt`, flag 2048, `utf8`. Both entries contain the same raw filename bytes `c3 a9 2e 74 78 74` but have different general-purpose bit 11 settings. There is no Unicode Path extra field. Run the built public workflow: ```sh node scripts/rea.mjs inspect-artifact "$PWD/names.zip" --json ``` Actual exit 1, `artifact_operation_failed`, `Artifact path collision: é.txt`. With only the first member, the command exits 0 but incorrectly inventories `é.txt` instead of `├⌐.txt`. ASCII, CP437 byte 0x82 and explicitly UTF-8 flagged controls all succeed. `ZipArtifactReader` supplies REA's CP437 decoder but leaves `filenameEncoding` unspecified. The installed zip.js reader then guesses UTF-8 before invoking that decoder. The name owner needs an explicit legacy fallback while retaining bit-11 UTF-8 and valid Unicode Path overrides. Path/collision, CRC, overlap and size validation should stay intact. This is within the existing ZIP support and the declared behavior of merged #2365, whose distinct change handled ASCII control characters. That PR does not disable the encoding guess. [PKWARE APPNOTE Appendix D.1–D.2](https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) specifies the legacy character set when bit 11 is unset and UTF-8 when it is set. Current targeted issue/PR searches for CP437, filenameEncoding and UTF8 flag found no open fix for this case.

bsakaguchi

2026-10-15 18:31:37 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:38 +00:00

Author

Owner

@gongfpp commented on GitHub (Oct 14, 2026):

Planning to fix this.

ZipArtifactReader passes its own CP437 decoder but leaves zip.js's filenameEncoding unset, so zip.js decides the legacy name itself: zip-reader.js computes filenameUTF8 = bit11 || (!filenameEncoding && isUTF8Text(raw)). An unflagged name whose bytes also form valid UTF-8 is therefore decoded as UTF-8 before REA's decoder sees it, which renames the member and makes the two spellings collide.

Naming the legacy encoding in the reader options keeps bit 11 and a valid Unicode Path extra field authoritative, and leaves the existing strict-UTF-8 and CP437 decoders in place. Path, collision, CRC, overlap and size validation are unchanged.

@gongfpp commented on GitHub (Oct 14, 2026): Planning to fix this. `ZipArtifactReader` passes its own CP437 decoder but leaves zip.js's `filenameEncoding` unset, so zip.js decides the legacy name itself: `zip-reader.js` computes `filenameUTF8 = bit11 || (!filenameEncoding && isUTF8Text(raw))`. An unflagged name whose bytes also form valid UTF-8 is therefore decoded as UTF-8 before REA's decoder sees it, which renames the member and makes the two spellings collide. Naming the legacy encoding in the reader options keeps bit 11 and a valid Unicode Path extra field authoritative, and leaves the existing strict-UTF-8 and CP437 decoders in place. Path, collision, CRC, overlap and size validation are unchanged.

bsakaguchi referenced this issue

2026-10-17 21:09:11 +00:00

[PR #1161] [MERGED] fix(artifacts): report wrong target kinds as unsupported targets #2696

bsakaguchi referenced this issue

2026-10-17 21:09:12 +00:00

[PR #1171] [MERGED] fix(artifacts): keep cancellation and the caller path in extraction refusals #2697

bsakaguchi referenced this issue

2026-10-31 00:40:56 +00:00

[GH-ISSUE #5731] [Bug] binary_session advertises Apple artifact operations for unsupported ZIP and plist targets #3478