[GH-ISSUE #3273] SAZ metadata accepts contradictory XML encoding declarations #1168

New issue

Closed

opened 2026-10-15 18:31:42 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-15 18:31:42 +00:00

Owner

Originally created by @xu91102 on GitHub (Oct 14, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/3273

Reproduction and observed result

On main 29d3dcb7ea85b3be37cd81e977ae4e09cebdd2fc, Node 24.18.0/npm 11.16.0 and locked dependencies, create an ordinary ZIP-based .saz containing a single raw/1_m.xml:

<?xml version="1.0" encoding="UTF-16"?><Session><Note>café</Note></Session>

Store those bytes as UTF-8, then run rea inspect-web-network-capture <archive.saz> saz --json. REA exits 0 and retains the Session metadata, even though the encoding declaration contradicts the bytes. The same operation also accepts BOM-marked UTF-16LE bytes declaring UTF-8. Python's byte-aware XML parser rejects both original byte inputs as incorrect encoding declarations.

Another fixture declares ISO-8859-1 but stores UTF-8-shaped bytes for café. The declaration-aware byte parser reads cafÃ©; REA instead reports café without a decoding limitation. This encoding is outside REA's existing UTF-8/BOM-UTF-16 metadata profile and should be refused rather than silently interpreted as a different charset.

Expected behavior and controls

Reject supported encodings with contradictory declarations and explicitly refuse unsupported declared encodings before producing successful metadata Evidence. Preserve consistent UTF-8 and BOM-marked UTF-16LE/BE metadata. All three supported positive controls currently pass, and actual Latin-1 bytes already fail the existing supported-profile gate. Every original archive remains byte-identical after inspection.

All records should remain validated before optional record selection; the same contradiction in an unselected second Session must not become a successful complete-input inspection. This is a passive local-data correctness issue, not a new live-Fiddler or decoder execution capability.

Cause and scope

SazMetadata chooses a decoder solely from the BOM/default UTF-8. Its existing SAX well-formedness check parses the resulting string but does not compare the parsed XML encoding declaration with that byte interpretation. Repair the shared byte/declaration owner before DOM projection; retain the existing malformed-character, DTD, budget, cancellation and cleanup policies.

XML 1.0 section 4.3.3 requires the byte encoding to agree with the declaration and permits refusing unsupported encodings. The existing metadata profile can stay narrow; no new charset support is needed.

Verified with actual compiled CLI and independently parsed original XML bytes on macOS arm64. No repair, Windows/Linux, MCP or current Fiddler UI export is claimed yet. Targeted live searches and the relevant #2884/#3082 diffs show no equivalent repair; closed #2601 concerns XML character/grammar validation and does not validate encoding declarations.

Originally created by @xu91102 on GitHub (Oct 14, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/3273 ## Reproduction and observed result On main `29d3dcb7ea85b3be37cd81e977ae4e09cebdd2fc`, Node 24.18.0/npm 11.16.0 and locked dependencies, create an ordinary ZIP-based `.saz` containing a single `raw/1_m.xml`: ```xml <?xml version="1.0" encoding="UTF-16"?><Session><Note>café</Note></Session> ``` Store those bytes as UTF-8, then run `rea inspect-web-network-capture <archive.saz> saz --json`. REA exits 0 and retains the Session metadata, even though the encoding declaration contradicts the bytes. The same operation also accepts BOM-marked UTF-16LE bytes declaring UTF-8. Python's byte-aware XML parser rejects both original byte inputs as incorrect encoding declarations. Another fixture declares ISO-8859-1 but stores UTF-8-shaped bytes for `café`. The declaration-aware byte parser reads `cafÃ©`; REA instead reports `café` without a decoding limitation. This encoding is outside REA's existing UTF-8/BOM-UTF-16 metadata profile and should be refused rather than silently interpreted as a different charset. ## Expected behavior and controls Reject supported encodings with contradictory declarations and explicitly refuse unsupported declared encodings before producing successful metadata Evidence. Preserve consistent UTF-8 and BOM-marked UTF-16LE/BE metadata. All three supported positive controls currently pass, and actual Latin-1 bytes already fail the existing supported-profile gate. Every original archive remains byte-identical after inspection. All records should remain validated before optional record selection; the same contradiction in an unselected second Session must not become a successful complete-input inspection. This is a passive local-data correctness issue, not a new live-Fiddler or decoder execution capability. ## Cause and scope `SazMetadata` chooses a decoder solely from the BOM/default UTF-8. Its existing SAX well-formedness check parses the resulting string but does not compare the parsed XML encoding declaration with that byte interpretation. Repair the shared byte/declaration owner before DOM projection; retain the existing malformed-character, DTD, budget, cancellation and cleanup policies. [XML 1.0 section 4.3.3](https://www.w3.org/TR/xml/#charencoding) requires the byte encoding to agree with the declaration and permits refusing unsupported encodings. The existing metadata profile can stay narrow; no new charset support is needed. Verified with actual compiled CLI and independently parsed original XML bytes on macOS arm64. No repair, Windows/Linux, MCP or current Fiddler UI export is claimed yet. Targeted live searches and the relevant #2884/#3082 diffs show no equivalent repair; closed #2601 concerns XML character/grammar validation and does not validate encoding declarations.

bsakaguchi

2026-10-15 18:31:42 +00:00

closed this issue
added the
bug
label

bsakaguchi commented

2026-10-15 18:31:43 +00:00

Author

Owner

@IvenKooLab commented on GitHub (Oct 14, 2026):

Claiming this for a fix.

Scope I am taking — validate the declared XML encoding against the actual byte shape before the SAZ session metadata is decoded, at the producing reader:

a UTF-16-family declaration over non-BOM UTF-8 bytes, and a UTF-8 declaration over BOM-marked UTF-16, fail the metadata extraction as a contradictory declaration instead of decoding silently;
declared encodings outside the existing UTF-8/BOM-UTF-16 metadata profile (e.g. ISO-8859-1) are refused explicitly, naming the declared encoding, instead of being read as UTF-8;
regression fixtures for the three reported shapes plus a control that consistent declarations still load.

Validation will use synthetic SAZ archives with the exact byte inputs from the report.

@IvenKooLab commented on GitHub (Oct 14, 2026): Claiming this for a fix. Scope I am taking — validate the declared XML encoding against the actual byte shape before the SAZ session metadata is decoded, at the producing reader: - a UTF-16-family declaration over non-BOM UTF-8 bytes, and a UTF-8 declaration over BOM-marked UTF-16, fail the metadata extraction as a contradictory declaration instead of decoding silently; - declared encodings outside the existing UTF-8/BOM-UTF-16 metadata profile (e.g. ISO-8859-1) are refused explicitly, naming the declared encoding, instead of being read as UTF-8; - regression fixtures for the three reported shapes plus a control that consistent declarations still load. Validation will use synthetic SAZ archives with the exact byte inputs from the report.

bsakaguchi commented

2026-10-15 18:31:43 +00:00

Author

Owner

@morluto commented on GitHub (Oct 14, 2026):

Resolved on main by 1250aaedd.

SAZ metadata now validates the declared XML encoding against the original byte/BOM decoding before projection, including contradictory and unsupported encoding controls.

Rechecked on main b8510ee00 in the current focused run: 17 test files passed, 485 tests passed and one platform-dependent test skipped. Relevant regression coverage: src/domain/propertyListXmlText.test.ts, src/browser/history/SazMetadata.test.ts and tests/boundary/browser/sazWebCapture.test.ts.

@morluto commented on GitHub (Oct 14, 2026): Resolved on main by 1250aaedd. SAZ metadata now validates the declared XML encoding against the original byte/BOM decoding before projection, including contradictory and unsupported encoding controls. Rechecked on main b8510ee00 in the current focused run: 17 test files passed, 485 tests passed and one platform-dependent test skipped. Relevant regression coverage: src/domain/propertyListXmlText.test.ts, src/browser/history/SazMetadata.test.ts and tests/boundary/browser/sazWebCapture.test.ts.

bsakaguchi referenced this issue

2026-10-17 21:09:12 +00:00

[PR #1172] [MERGED] fix(artifacts): match lipo arm64e variant names to their FAT slices #2698

bsakaguchi referenced this issue

2026-10-31 00:42:43 +00:00

[GH-ISSUE #5982] [Bug] Unparseable lipo slice output becomes a generic artifact I/O failure #3650

bsakaguchi referenced this issue

2026-10-31 00:50:17 +00:00

[GH-ISSUE #7555] [Bug] inspect-signature rejects valid thin arm64e Mach-O files #4496

bsakaguchi referenced this issue

2026-10-31 00:57:16 +00:00

[PR #1982] [MERGED] fix(native): accept universal binaries with LIB64 x86_64 slices #4948