[PR #2804] [MERGED] fix(flutter): find snapshot headers that a read chunk boundary splits #5407

Closed
opened 2026-10-31 01:06:24 +00:00 by bsakaguchi · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/2804
Author: @Dragolone
Created: 10/13/2026
Status: ✅ Merged
Merged: 10/13/2026
Merged by: @morluto

Base: main ← Head: fix/flutter-snapshot-scan-chunk-boundary


📝 Commits (2)

  • c24f744 fix(flutter): find snapshot headers that a read chunk boundary splits
  • 743dbfb refactor(flutter): drop the snapshot scanner cursor the fix superseded

📊 Changes

2 files changed (+37 additions, -14 deletions)

View changed files

📝 src/flutter/FlutterPayloadScan.test.ts (+31 -0)
📝 src/flutter/FlutterPayloadScan.ts (+6 -14)

📄 Description

Summary

identify_flutter_build can omit a Dart snapshot hash, and report coverage: "complete" with a single agreed hash, depending only on where the APK reader's chunk boundaries fall.

Within one chunk, SnapshotHashScanner resumes four bytes after each magic. Across chunks it resumed after the whole 52-byte magic-to-hash record and dropped the carried bytes. A magic starting 49–51 bytes after a rejected (non-hex) record was therefore lost when its four bytes straddled a read boundary. This PR resumes after the magic in both cases.

No issue is linked: I searched issues and PRs for SnapshotHashScanner, FlutterPayloadScan, "snapshot hash chunk boundary", "snapshot magic" and identify_flutter_build and found no report. The #1582 review fixed chunk-boundary cases in the labeled-string scanner, not this one. Open PR #2334 touches DartElf.ts, not this scanner.

Problem and expected behavior

Smallest trigger: a stray magic f5 f5 dc dc whose hash field is not lowercase hex, followed 50 bytes later by a real snapshot section. Fed as one chunk, the scanner finds 1 source. Split at byte 52, it finds 0.

End to end: an APK whose lib/arm64-v8a/libapp.so holds the VM and isolate sections (hash 65817c30…), a stray magic at 65484, and a third section with hash 2222… at 65534. The third section's magic straddles zip.js's 64 KiB read boundary.

Violated contract: docs/flutter-analysis.md says that when snapshot hashes disagree, all candidates are kept, snapshot_hash is null, and coverage is partial rather than silently picking one. With the bug, the conflicting hash is dropped and the result says complete.

Expected: the result does not depend on chunking, so the APK above reports both candidates, a null hash, and partial coverage.

Change and scope

The cross-chunk resume point now matches the in-chunk one: right after the processed magic. After a complete record, the carried tail is shorter than a magic. A pending magic has not been processed yet. So each push can rescan its carried bytes from the start without counting a record twice. That made the absolute cursor (#absoluteScanFrom) and its #base offset unnecessary, and both are removed (thanks to the Codex review). No other scanner behavior changes. Not included: the labeled-string scanner and DartElf.ts.

Contribution scope: an ordinary static format-inspection correctness fix within existing boundaries; a maintainer scope decision is not applicable.

Contract and boundary impact

  • Semantic owner and earliest changed stage: SnapshotHashScanner.push in src/flutter/FlutterPayloadScan.ts.
  • Contribution scope fit and linked maintainer decision: in scope (format inspection, evidence reporting); not applicable.
  • CLI and MCP/tool-catalog contract: no schema change. snapshot_hash_sources, snapshot_hash_candidates, snapshot_hash and coverage now match the single-chunk result for affected inputs.
  • Provider, bridge, target-format, or platform compatibility: none.
  • Evidence, artifact, provenance, or reconstruction contract: fixes a case where a disagreement was silently omitted while coverage said complete.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: src/flutter/FlutterPayloadScan.test.ts — "finds a section %i bytes after a rejected record at any chunking", run for offsets 49, 50 and 51. It splits the payload at every byte and also feeds it one byte at a time. Every result must equal the single-chunk result.
  • Base reproduction: on 91830f1e0 all three cases fail with AssertionError: split at 52: expected { sources: +0, candidates: [], …(1) } to deeply equal { sources: 1, …(2) }.
  • User-visible output: FlutterBuildProvider identify_flutter_build on the APK described above (a local test that was not committed):
    • Before: { coverage: 'complete', sources: 2, candidates: [ '65817c30a78bb44c3dc3771876b6010a' ], hash: '65817c30a78bb44c3dc3771876b6010a' }
    • After: { coverage: 'partial', sources: 3, candidates: [ '65817c30a78bb44c3dc3771876b6010a', '22222222222222222222222222222222' ], hash: null }
  • Remaining proof gaps: the trigger needs a magic within 49–51 bytes after a rejected record and a read boundary inside the second magic. I have not observed it in a real Flutter APK.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

macOS (Apple silicon), Node 24.19.0.

  • npx vitest run src/flutter/FlutterPayloadScan.test.ts against the base scanner — the three new cases fail as quoted above (14 others pass).
  • npx vitest run src/flutter/ with the fix — 2 files, 31 tests passed.
  • A local split fuzz (not committed): 50,000 random payloads with clustered magics and valid hashes, each fed in random 1–80-byte chunks and compared with a single push. The base had 1 mismatch; the fix, before and after the cursor removal, had 0.
  • The end-to-end identify_flutter_build comparison above, run against the base and the fix.
  • npm run typecheck — passed.
  • npx oxfmt --check on both changed files — passed.
  • npm run verify:module-boundaries — passed.
  • npm run lint — the only error is tests/boundary/providers/ghidra/ghidraLauncher.test.ts:431 (max-lines-per-function), which this PR does not touch. The changed files have only the existing no-control-regex warnings.

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no change; pure in-memory byte scanning.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/2804 **Author:** [@Dragolone](https://github.com/Dragolone) **Created:** 10/13/2026 **Status:** ✅ Merged **Merged:** 10/13/2026 **Merged by:** [@morluto](https://github.com/morluto) **Base:** `main` ← **Head:** `fix/flutter-snapshot-scan-chunk-boundary` --- ### 📝 Commits (2) - [`c24f744`](https://github.com/morluto/rea/commit/c24f7443f3f3faf42cbbf1b1c5734816b62ca808) fix(flutter): find snapshot headers that a read chunk boundary splits - [`743dbfb`](https://github.com/morluto/rea/commit/743dbfb6569a017ea9bbde7c046c06cc52220e29) refactor(flutter): drop the snapshot scanner cursor the fix superseded ### 📊 Changes **2 files changed** (+37 additions, -14 deletions) <details> <summary>View changed files</summary> 📝 `src/flutter/FlutterPayloadScan.test.ts` (+31 -0) 📝 `src/flutter/FlutterPayloadScan.ts` (+6 -14) </details> ### 📄 Description ## Summary `identify_flutter_build` can omit a Dart snapshot hash, and report `coverage: "complete"` with a single agreed hash, depending only on where the APK reader's chunk boundaries fall. Within one chunk, `SnapshotHashScanner` resumes four bytes after each magic. Across chunks it resumed after the whole 52-byte magic-to-hash record and dropped the carried bytes. A magic starting 49–51 bytes after a rejected (non-hex) record was therefore lost when its four bytes straddled a read boundary. This PR resumes after the magic in both cases. No issue is linked: I searched issues and PRs for `SnapshotHashScanner`, `FlutterPayloadScan`, "snapshot hash chunk boundary", "snapshot magic" and `identify_flutter_build` and found no report. The #1582 review fixed chunk-boundary cases in the labeled-string scanner, not this one. Open PR #2334 touches `DartElf.ts`, not this scanner. ## Problem and expected behavior Smallest trigger: a stray magic `f5 f5 dc dc` whose hash field is not lowercase hex, followed 50 bytes later by a real snapshot section. Fed as one chunk, the scanner finds 1 source. Split at byte 52, it finds 0. End to end: an APK whose `lib/arm64-v8a/libapp.so` holds the VM and isolate sections (hash `65817c30…`), a stray magic at 65484, and a third section with hash `2222…` at 65534. The third section's magic straddles zip.js's 64 KiB read boundary. Violated contract: `docs/flutter-analysis.md` says that when snapshot hashes disagree, all candidates are kept, `snapshot_hash` is null, and coverage is partial rather than silently picking one. With the bug, the conflicting hash is dropped and the result says complete. Expected: the result does not depend on chunking, so the APK above reports both candidates, a null hash, and partial coverage. ## Change and scope The cross-chunk resume point now matches the in-chunk one: right after the processed magic. After a complete record, the carried tail is shorter than a magic. A pending magic has not been processed yet. So each push can rescan its carried bytes from the start without counting a record twice. That made the absolute cursor (`#absoluteScanFrom`) and its `#base` offset unnecessary, and both are removed (thanks to the Codex review). No other scanner behavior changes. Not included: the labeled-string scanner and `DartElf.ts`. Contribution scope: an ordinary static format-inspection correctness fix within existing boundaries; a maintainer scope decision is not applicable. ## Contract and boundary impact - Semantic owner and earliest changed stage: `SnapshotHashScanner.push` in `src/flutter/FlutterPayloadScan.ts`. - Contribution scope fit and linked maintainer decision: in scope (format inspection, evidence reporting); not applicable. - CLI and MCP/tool-catalog contract: no schema change. `snapshot_hash_sources`, `snapshot_hash_candidates`, `snapshot_hash` and `coverage` now match the single-chunk result for affected inputs. - Provider, bridge, target-format, or platform compatibility: none. - Evidence, artifact, provenance, or reconstruction contract: fixes a case where a disagreement was silently omitted while coverage said complete. - Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable. - Generated metadata (`docs/public/product-catalog.json`), package, or installation impact: none. Expected effects (check known effects; use documentation-only on its own). Unchecked items do not establish absence; leave uncertainties visible. - [x] Read existing files or captured data. - [ ] Launch or attach to a process. - [ ] Inject instrumentation or run scripts/code. - [ ] Change process memory, registers, files, or configuration. - [ ] Contact network services. - [ ] Recover protected code or work around protections. - [ ] Documentation or usability only; no target interaction. - [ ] Not sure yet; investigation or maintainer guidance needed. ## Evidence and regression coverage - Tests added or updated: `src/flutter/FlutterPayloadScan.test.ts` — "finds a section %i bytes after a rejected record at any chunking", run for offsets 49, 50 and 51. It splits the payload at every byte and also feeds it one byte at a time. Every result must equal the single-chunk result. - Base reproduction: on `91830f1e0` all three cases fail with `AssertionError: split at 52: expected { sources: +0, candidates: [], …(1) } to deeply equal { sources: 1, …(2) }`. - User-visible output: `FlutterBuildProvider` `identify_flutter_build` on the APK described above (a local test that was not committed): - Before: `{ coverage: 'complete', sources: 2, candidates: [ '65817c30a78bb44c3dc3771876b6010a' ], hash: '65817c30a78bb44c3dc3771876b6010a' }` - After: `{ coverage: 'partial', sources: 3, candidates: [ '65817c30a78bb44c3dc3771876b6010a', '22222222222222222222222222222222' ], hash: null }` - Remaining proof gaps: the trigger needs a magic within 49–51 bytes after a rejected record and a read boundary inside the second magic. I have not observed it in a real Flutter APK. For evidence-bearing changes: - [x] Observed, derived, and inferred claims remain distinguishable. - [x] Artifact identity, source provenance, and failed attempts remain preserved. - [x] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible. ## Validation performed macOS (Apple silicon), Node 24.19.0. - `npx vitest run src/flutter/FlutterPayloadScan.test.ts` against the base scanner — the three new cases fail as quoted above (14 others pass). - `npx vitest run src/flutter/` with the fix — 2 files, 31 tests passed. - A local split fuzz (not committed): 50,000 random payloads with clustered magics and valid hashes, each fed in random 1–80-byte chunks and compared with a single push. The base had 1 mismatch; the fix, before and after the cursor removal, had 0. - The end-to-end `identify_flutter_build` comparison above, run against the base and the fix. - `npm run typecheck` — passed. - `npx oxfmt --check` on both changed files — passed. - `npm run verify:module-boundaries` — passed. - `npm run lint` — the only error is `tests/boundary/providers/ghidra/ghidraLauncher.test.ts:431` (`max-lines-per-function`), which this PR does not touch. The changed files have only the existing `no-control-regex` warnings. ## Compatibility, safety, and release - Breaking changes or migration steps: none. - Real Hopper/Ghidra, browser, or OS coverage: not applicable. - Package or release metadata impact: none. - Security, privacy, process, or containment review: no change; pure in-memory byte scanning. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
bsakaguchi 2026-10-31 01:06:24 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
morluto/rea#5407
No description provided.