[PR #218] [CLOSED] feat: add mincore snapshot profiling and KVM prefault #259

New issue

Closed

opened 2026-10-29 14:21:34 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:21:34 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/218
Author: @Hygge-Gezelligheid
Created: 8/25/2026
Status: ❌ Closed

Base: main ← Head: agent/idle-page-prefault-mvp

📝 Commits (2)

993795d feat: add mincore snapshot prefault
7fecb35 test: cover restore prefault fallbacks

📊 Changes

82 files changed (+2908 additions, -109 deletions)

View changed files

📝 Cargo.lock (+1 -1)
📝 Cargo.toml (+1 -1)
📝 Makefile (+3 -1)
📝 config/default.toml (+17 -0)
📝 crates/linux-cap/src/lib.rs (+6 -0)
📝 docs/src/getting-started/quickstart.md (+2 -1)
📝 docs/src/troubleshooting/common-issues.md (+15 -0)
📝 src/bin/server.rs (+6 -0)
📝 src/cfg.rs (+57 -0)
📝 src/privileges.rs (+100 -5)
➕ src/sandbox/firecracker/idle_page_tracking.rs (+738 -0)
📝 src/sandbox/firecracker/instance.rs (+257 -2)
📝 src/sandbox/firecracker/manifest.rs (+393 -1)
➕ src/sandbox/firecracker/mincore_tracking.rs (+302 -0)
📝 src/sandbox/firecracker/mod.rs (+8 -1)
➕ src/sandbox/firecracker/prefault.rs (+219 -0)
📝 src/sandbox/firecracker/sandbox.rs (+474 -25)
📝 src/sandbox/mod.rs (+1 -0)
📝 src/sandbox/ublk/device.rs (+41 -0)
📝 src/template/runner.rs (+37 -7)

...and 62 more files

📄 Description

Summary

Add opt-in snapshot working-set profiling based on mincore, persist guest-physical-address working-set metadata, and pre-fault that metadata through Firecracker v1.15.1-patch-v2 before a restore resumes. The default configuration remains disabled.

This Draft PR consists of 993795d1bc5ff4e563e68e7f39440cbe866bf7c9 (feat: add mincore snapshot prefault) and 7fecb35a57351450321e844d3ca734d2057ba164 (test: cover restore prefault fallbacks).

Firecracker dependency scope

Uses the v2 OpenAPI client contract from Firecracker Draft PR #20: PUT /vm/pre-fault-memory, dirty/resident ranges, and guest-memory regions with guest_phys_addr.
config/deps_manifest.toml deliberately remains at the released 1.15.1-patch-v1 asset. No v2 tag or Release exists yet, so changing it would make the installer resolve an unpublished dependency.
This PR does not change Firecracker snapshot serialization or the upstream snapshot data version.

Behavior and safety

Default tracker is mincore; Linux idle-page tracking is retained only as explicit tracker = "idle-page" opt-in. The normal service/capability runner keeps only CAP_NET_ADMIN and CAP_SYS_ADMIN; CAP_DAC_OVERRIDE is required only for explicit idle-page mode.
Profiling uses a per-run exclusive UBLK memory device, disables dirty-page tracking only in its cloned profiling restore configuration, records a paused baseline after snapshot load, resumes for the profiling interval, pauses again, and writes only complete validated metadata. Normal restore behavior and snapshot serialization are unchanged.
The mincore path rejects a host with configured swap before profiling; it skips metadata publication rather than changing host swap. Restore pre-faulting is best-effort: invalid/missing metadata or API failures warn and resume normally.
Ranges are validated for alignment, containment in one memory-image region, checked GPA arithmetic, configured byte/range/RAM budgets, then sorted, deduplicated, and coalesced only when GPA-contiguous.

Compatibility evidence

A server runtime check created a paused file-memory snapshot with the existing Firecracker v1.15.1 artifact (snapshot version v9.0.0) and loaded it with the v2 candidate, also v9.0.0. The v2 VM accepted guest-memory-regions, dirty-memory-ranges, and a valid KVM pre-fault request after loading the old snapshot. This is direct old-v1-to-v2 snapshot compatibility evidence for the file-memory route; it is not a full OverlayBD/AgentENV fixture run.

Validation

Passed on agentenv-lab after loading its existing Rust environment (rustc/cargo 1.95.0):

cargo fmt --all -- --check
git diff --check and committed git diff --check HEAD^ HEAD
cargo check --all-targets
cargo clippy --all-targets -- -D warnings
make test-unit: AgentENV 775 passed / 6 ignored, linux-cap 4 passed, capability-runner ignored subset 4 passed, UBLK 8 passed, UBLK daemon 46 passed, installer/capability scripts passed
After the follow-up test commit: cargo check --all-targets, cargo clippy --all-targets -- -D warnings, and full AgentENV library tests (778 passed / 6 ignored) passed.
Focused fake-Unix-socket API tests verify GPA parsing, dirty ranges, exact pre-fault request JSON, empty-range suppression, guest-regions-before-pre-fault ordering, endpoint-unavailable fallback, and no API calls for empty metadata.
Firecracker v2 server validation: pre-fault unit/API parser/snapshot/seccomp tests and direct KVM endpoint checks passed. Both x86_64 and aarch64 musl candidate archives now contain only firecracker and cpu-template-helper; the aarch64 artifact passed ELF/static-link inspection.

Not executed / limitations

A complete AgentENV OverlayBD rootfs template build, actual mincore metadata harvest, restored AgentENV fixture run, and cold-restore A/B performance measurement were not executed. The test host reports SwapTotal: 8388604 kB; the implementation intentionally refuses profiling in that condition and does not alter host swap. Suitable no-swap fixture infrastructure is still required.
Cross-compiled/static ARM checks are not native ARM64/KVM runtime verification.
The profiling execution uses the normal sandbox networking policy; it does not add a dedicated egress-deny rule.

Review focus

Please review the generated v2 OpenAPI client update, the explicit capability split for idle-page mode, mincore range/GPA validation, and the pause/resume/error paths around profiling and restore pre-faulting.

No Firecracker tag, AgentENV tag, or Release has been created or published.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/218 **Author:** [@Hygge-Gezelligheid](https://github.com/Hygge-Gezelligheid) **Created:** 8/25/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `agent/idle-page-prefault-mvp` --- ### 📝 Commits (2) - [`993795d`](https://github.com/kvcache-ai/AgentENV/commit/993795d1bc5ff4e563e68e7f39440cbe866bf7c9) feat: add mincore snapshot prefault - [`7fecb35`](https://github.com/kvcache-ai/AgentENV/commit/7fecb35a57351450321e844d3ca734d2057ba164) test: cover restore prefault fallbacks ### 📊 Changes **82 files changed** (+2908 additions, -109 deletions) <details> <summary>View changed files</summary> 📝 `Cargo.lock` (+1 -1) 📝 `Cargo.toml` (+1 -1) 📝 `Makefile` (+3 -1) 📝 `config/default.toml` (+17 -0) 📝 `crates/linux-cap/src/lib.rs` (+6 -0) 📝 `docs/src/getting-started/quickstart.md` (+2 -1) 📝 `docs/src/troubleshooting/common-issues.md` (+15 -0) 📝 `src/bin/server.rs` (+6 -0) 📝 `src/cfg.rs` (+57 -0) 📝 `src/privileges.rs` (+100 -5) ➕ `src/sandbox/firecracker/idle_page_tracking.rs` (+738 -0) 📝 `src/sandbox/firecracker/instance.rs` (+257 -2) 📝 `src/sandbox/firecracker/manifest.rs` (+393 -1) ➕ `src/sandbox/firecracker/mincore_tracking.rs` (+302 -0) 📝 `src/sandbox/firecracker/mod.rs` (+8 -1) ➕ `src/sandbox/firecracker/prefault.rs` (+219 -0) 📝 `src/sandbox/firecracker/sandbox.rs` (+474 -25) 📝 `src/sandbox/mod.rs` (+1 -0) 📝 `src/sandbox/ublk/device.rs` (+41 -0) 📝 `src/template/runner.rs` (+37 -7) _...and 62 more files_ </details> ### 📄 Description ## Summary Add opt-in snapshot working-set profiling based on `mincore`, persist guest-physical-address working-set metadata, and pre-fault that metadata through Firecracker v1.15.1-patch-v2 before a restore resumes. The default configuration remains disabled. This Draft PR consists of `993795d1bc5ff4e563e68e7f39440cbe866bf7c9` (`feat: add mincore snapshot prefault`) and `7fecb35a57351450321e844d3ca734d2057ba164` (`test: cover restore prefault fallbacks`). ## Firecracker dependency scope - Uses the v2 OpenAPI client contract from Firecracker Draft PR #20: `PUT /vm/pre-fault-memory`, dirty/resident ranges, and guest-memory regions with `guest_phys_addr`. - `config/deps_manifest.toml` deliberately remains at the released `1.15.1-patch-v1` asset. No v2 tag or Release exists yet, so changing it would make the installer resolve an unpublished dependency. - This PR does not change Firecracker snapshot serialization or the upstream snapshot data version. ## Behavior and safety - Default tracker is `mincore`; Linux idle-page tracking is retained only as explicit `tracker = "idle-page"` opt-in. The normal service/capability runner keeps only `CAP_NET_ADMIN` and `CAP_SYS_ADMIN`; `CAP_DAC_OVERRIDE` is required only for explicit idle-page mode. - Profiling uses a per-run exclusive UBLK memory device, disables dirty-page tracking only in its cloned profiling restore configuration, records a paused baseline after snapshot load, resumes for the profiling interval, pauses again, and writes only complete validated metadata. Normal restore behavior and snapshot serialization are unchanged. - The mincore path rejects a host with configured swap before profiling; it skips metadata publication rather than changing host swap. Restore pre-faulting is best-effort: invalid/missing metadata or API failures warn and resume normally. - Ranges are validated for alignment, containment in one memory-image region, checked GPA arithmetic, configured byte/range/RAM budgets, then sorted, deduplicated, and coalesced only when GPA-contiguous. ## Compatibility evidence A server runtime check created a paused file-memory snapshot with the existing Firecracker v1.15.1 artifact (`snapshot version v9.0.0`) and loaded it with the v2 candidate, also `v9.0.0`. The v2 VM accepted guest-memory-regions, dirty-memory-ranges, and a valid KVM pre-fault request after loading the old snapshot. This is direct old-v1-to-v2 snapshot compatibility evidence for the file-memory route; it is not a full OverlayBD/AgentENV fixture run. ## Validation Passed on `agentenv-lab` after loading its existing Rust environment (rustc/cargo 1.95.0): - `cargo fmt --all -- --check` - `git diff --check` and committed `git diff --check HEAD^ HEAD` - `cargo check --all-targets` - `cargo clippy --all-targets -- -D warnings` - `make test-unit`: AgentENV 775 passed / 6 ignored, linux-cap 4 passed, capability-runner ignored subset 4 passed, UBLK 8 passed, UBLK daemon 46 passed, installer/capability scripts passed - After the follow-up test commit: `cargo check --all-targets`, `cargo clippy --all-targets -- -D warnings`, and full AgentENV library tests (778 passed / 6 ignored) passed. - Focused fake-Unix-socket API tests verify GPA parsing, dirty ranges, exact pre-fault request JSON, empty-range suppression, guest-regions-before-pre-fault ordering, endpoint-unavailable fallback, and no API calls for empty metadata. - Firecracker v2 server validation: pre-fault unit/API parser/snapshot/seccomp tests and direct KVM endpoint checks passed. Both x86_64 and aarch64 musl candidate archives now contain only `firecracker` and `cpu-template-helper`; the aarch64 artifact passed ELF/static-link inspection. ## Not executed / limitations - A complete AgentENV OverlayBD rootfs template build, actual mincore metadata harvest, restored AgentENV fixture run, and cold-restore A/B performance measurement were not executed. The test host reports `SwapTotal: 8388604 kB`; the implementation intentionally refuses profiling in that condition and does not alter host swap. Suitable no-swap fixture infrastructure is still required. - Cross-compiled/static ARM checks are not native ARM64/KVM runtime verification. - The profiling execution uses the normal sandbox networking policy; it does not add a dedicated egress-deny rule. ## Review focus Please review the generated v2 OpenAPI client update, the explicit capability split for idle-page mode, mincore range/GPA validation, and the pause/resume/error paths around profiling and restore pre-faulting. No Firecracker tag, AgentENV tag, or Release has been created or published. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:21:34 +00:00