[GH-ISSUE #40] LSMT mapped reads silently zero-fill truncated backing data #16

New issue

Closed

opened 2026-10-29 14:18:24 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:18:24 +00:00

Owner

Originally created by @morluto on GitHub (Jul 28, 2026).
Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/40

Problem

The readonly and readwrite LSMT mapped-read paths treat a short physical read from a backing layer as valid data followed by zeros.

That makes a truncated or incomplete mapped extent indistinguishable from an intentional sparse hole. Because OverlayBD backs AgentENV root filesystems and memory snapshots, callers can receive invented zero bytes instead of an I/O error.

mapping lookup
  +-- hole / zero mapping ------------> fill with zeros
  |
  `-- mapped extent
          |
          v
      read remaining bytes <----------+
          |                           |
          +-- bytes, range incomplete +
          +-- range complete --------> return data
          `-- early EOF -------------> return error

Current behavior

LSMTReadOnlyFile::read_internal_into_generic and LSMTFile::read_mappings_into_generic fill the unread portion of the destination buffer with zeros when a mapped physical read returns fewer bytes than requested.

Explicit holes and zero mappings are handled separately, so a short read from a mapped extent should not use the same zero-fill behavior.

Expected behavior

Retry legal partial reads until the mapped extent is complete.
Return an error if EOF occurs before the extent is complete.
Continue zero-filling explicit holes and zero mappings.

Impact

A truncated local, registry-backed, or otherwise incomplete layer can be exposed as apparently valid filesystem or VM-memory data. Surfacing the read failure preserves the distinction between sparse data and backing-layer corruption.

Originally created by @morluto on GitHub (Jul 28, 2026). Original GitHub issue: https://github.com/kvcache-ai/AgentENV/issues/40 ## Problem The readonly and readwrite LSMT mapped-read paths treat a short physical read from a backing layer as valid data followed by zeros. That makes a truncated or incomplete mapped extent indistinguishable from an intentional sparse hole. Because OverlayBD backs AgentENV root filesystems and memory snapshots, callers can receive invented zero bytes instead of an I/O error. ```text mapping lookup +-- hole / zero mapping ------------> fill with zeros | `-- mapped extent | v read remaining bytes <----------+ | | +-- bytes, range incomplete + +-- range complete --------> return data `-- early EOF -------------> return error ``` ## Current behavior `LSMTReadOnlyFile::read_internal_into_generic` and `LSMTFile::read_mappings_into_generic` fill the unread portion of the destination buffer with zeros when a mapped physical read returns fewer bytes than requested. Explicit holes and zero mappings are handled separately, so a short read from a mapped extent should not use the same zero-fill behavior. ## Expected behavior - Retry legal partial reads until the mapped extent is complete. - Return an error if EOF occurs before the extent is complete. - Continue zero-filling explicit holes and zero mappings. ## Impact A truncated local, registry-backed, or otherwise incomplete layer can be exposed as apparently valid filesystem or VM-memory data. Surfacing the read failure preserves the distinction between sparse data and backing-layer corruption.

bsakaguchi closed this issue

2026-10-29 14:18:24 +00:00

bsakaguchi commented

2026-10-29 14:18:25 +00:00

Author

Owner

@yingdi-shan commented on GitHub (Jul 29, 2026):

Thank you so much! Could you please share the steps to reproduce this issue?

@yingdi-shan commented on GitHub (Jul 29, 2026): Thank you so much! Could you please share the steps to reproduce this issue?

bsakaguchi commented

2026-10-29 14:18:25 +00:00

Author

Owner

@morluto commented on GitHub (Jul 29, 2026):

Yes. This is reproduced by the focused OverlayBD unit test in #43; it does not require starting AgentENV or using KVM.

gh pr checkout 43
cargo test -p overlaybd --lib \
  test_lsmt_rejects_truncated_mapped_reads -- --nocapture

The test creates a 4 KiB mapped extent backed by only 3968 physical bytes and exercises both the read-only and read-write LSMT paths.

I also ran the test against the parent implementation while retaining the regression test. The old implementation returned Ok and left the missing 128-byte tail zero-filled; the fixed implementation returns unexpected EOF.

@morluto commented on GitHub (Jul 29, 2026): Yes. This is reproduced by the focused OverlayBD unit test in #43; it does not require starting AgentENV or using KVM. ```bash gh pr checkout 43 cargo test -p overlaybd --lib \ test_lsmt_rejects_truncated_mapped_reads -- --nocapture ``` The test creates a 4 KiB mapped extent backed by only 3968 physical bytes and exercises both the read-only and read-write LSMT paths. I also ran the test against the parent implementation while retaining the regression test. The old implementation returned `Ok` and left the missing 128-byte tail zero-filled; the fixed implementation returns `unexpected EOF`.

bsakaguchi referenced this issue

2026-10-29 14:19:55 +00:00

[PR #19] [MERGED] feat: expose snapshot rootfs image references #121

bsakaguchi referenced this issue

2026-10-29 14:20:01 +00:00

[PR #32] [CLOSED] fix(snapshot): isolate registry publications on delete #130