kvcache-ai/AgentENV

Fork 0

mirror of https://github.com/kvcache-ai/AgentENV.git synced 2026-11-05 01:17:36 +00:00

[PR #133] feat(snapshot): Advance memory snapshot epochs for linear incremental OverlayBD chains #198

New issue

Open

opened 2026-10-29 14:20:56 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:20:56 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/133
Author: @xeonliu
Created: 8/6/2026
Status: 🔄 Open

Base: main ← Head: feat/incrckpt

📝 Commits (2)

4d217a8 docs: clarify KVM access prerequisite
55d040f feat(snapshot): advance live memory parent after capture

📊 Changes

3 files changed (+81 additions, -11 deletions)

View changed files

📝 crates/benchmarks/benches/snapshot_benchmark.rs (+7 -0)
📝 docs/src/concepts/snapshots.md (+6 -0)
📝 src/sandbox/firecracker/sandbox.rs (+68 -11)

📄 Description

What

Introduce an explicit live memory parent for repeated non-terminal Firecracker captures.

After a locally successful capture, FirecrackerSandbox advances
current_memory_parent to the newly written mem_image.json. The next capture
inherits that image instead of rebuilding from the immutable launch-time parent.

This Draft also:

Keeps launch provenance separate from the mutable current parent.
Adds a unit test for the parent transition.
Documents the linear capture behavior.
Records follow-up TODOs for transactional epochs, artifact ownership, fork handling, and benchmarks.

Why

Repeated captures currently use the launch-time memory image as their parent.
This cannot support true per-checkpoint dirty-page epochs because resetting the
dirty set after one capture would allow later captures to omit earlier changes.

For example:

S0 -> write A -> capture S1 -> write B -> capture S2

The required memory lineage is:

S1 = S0 + delta(A)
S2 = S0 + delta(A) + delta(B)

This change introduces and advances the current parent needed to build that
linear chain.

Related to #120.

This remains a Draft and does not yet close the issue.

Scope and non-goals

Included:

Explicit current_memory_parent state in FirecrackerSandbox.
Initialization from the launch-time snapshot.
Parent advancement after a locally successful non-terminal capture.
Use of the current parent when constructing the next memory image.
Focused unit coverage and documentation.
Concrete benchmark TODOs.

Not yet included:

Transactional Firecracker dirty-epoch commit/abort.
Atomic and durable mem_image.json publication.
A dedicated lineage/lease object for artifact ownership.
Parent advancement for fork captures.
Chain compaction or garbage-collection policy.
Completed performance benchmarks.
Changes to the public API or OverlayBD format.

Design and behavior changes

launch remains immutable and records where the Firecracker process started.
current_memory_parent is initialized from it and advances independently.

The update currently occurs after pause_to_dir produces the local snapshot
configuration and before the VM resumes. Repository publication is not part of
this local parent transition.

Compatibility and operations

Public API or generated protocol: No changes.
Configuration or defaults: No changes.
Snapshot manifest, artifact layout, or storage format: No format changes; existing ordered OverlayBD lower layers are reused.
Upgrade and rollback: Existing snapshots remain readable. This Draft does not yet define mixed-version dirty-epoch behavior.
Host requirements, permissions, ports, or dependencies: No changes.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Commands and results:

cargo fmt --all -- --check
Passed.

git diff --check
Passed.

cargo test -p agentenv --lib current_memory_parent_advances_without_changing_launch_provenance
Did not compile on macOS because the Linux-only io-uring dependency requires
MAP_POPULATE, MADV_DONTFORK, cpu_set_t, and io_uring syscall constants.
The test executable was not run.

Skipped checks and reasons:

Full Rust and integration tests require a supported Linux host. No service or
generated-code checks are applicable.

Performance measurements remain TODO. The planned benchmark repeatedly rewrites
a fixed hot-page set and records:

Per-capture pause latency.
Emitted memory-layer bytes.
Chain depth.
Artifact-retention cost.
Behavior before and after the compaction threshold.
Results after 10 and 100 captures.

Risks and reviewer notes

This is intentionally an incomplete Draft.

The current parent update is not yet coupled to an atomic image-config commit
or transactional dirty-epoch acknowledgement. Fork captures also need to use
the same transition. Before merge, artifact ownership should move into an
explicit lineage/lease abstraction so every referenced local layer remains
pinned.

Linear chains retain obsolete versions of repeatedly written hot pages.
Compaction policy and benchmark results are required before enabling this
behavior in production.

The main review points are:

Whether current lineage should live directly in FirecrackerSandbox.
Whether local durable capture is the correct epoch commit boundary.
How fork captures participate in the same lineage.
What chain-depth and compaction limits are acceptable.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/133 **Author:** [@xeonliu](https://github.com/xeonliu) **Created:** 8/6/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `feat/incrckpt` --- ### 📝 Commits (2) - [`4d217a8`](https://github.com/kvcache-ai/AgentENV/commit/4d217a89b0e34be96636a7fa5d15cd272b3886b3) docs: clarify KVM access prerequisite - [`55d040f`](https://github.com/kvcache-ai/AgentENV/commit/55d040f886adefbd4069e84c55697c1b443599e2) feat(snapshot): advance live memory parent after capture ### 📊 Changes **3 files changed** (+81 additions, -11 deletions) <details> <summary>View changed files</summary> 📝 `crates/benchmarks/benches/snapshot_benchmark.rs` (+7 -0) 📝 `docs/src/concepts/snapshots.md` (+6 -0) 📝 `src/sandbox/firecracker/sandbox.rs` (+68 -11) </details> ### 📄 Description ## What Introduce an explicit live memory parent for repeated non-terminal Firecracker captures. After a locally successful capture, `FirecrackerSandbox` advances `current_memory_parent` to the newly written `mem_image.json`. The next capture inherits that image instead of rebuilding from the immutable launch-time parent. This Draft also: - Keeps launch provenance separate from the mutable current parent. - Adds a unit test for the parent transition. - Documents the linear capture behavior. - Records follow-up TODOs for transactional epochs, artifact ownership, fork handling, and benchmarks. ## Why Repeated captures currently use the launch-time memory image as their parent. This cannot support true per-checkpoint dirty-page epochs because resetting the dirty set after one capture would allow later captures to omit earlier changes. For example: ```text S0 -> write A -> capture S1 -> write B -> capture S2 The required memory lineage is: S1 = S0 + delta(A) S2 = S0 + delta(A) + delta(B) ``` This change introduces and advances the current parent needed to build that linear chain. ## Related issue Related to #120. This remains a Draft and does not yet close the issue. ## Scope and non-goals Included: - Explicit current_memory_parent state in FirecrackerSandbox. - Initialization from the launch-time snapshot. - Parent advancement after a locally successful non-terminal capture. - Use of the current parent when constructing the next memory image. - Focused unit coverage and documentation. - Concrete benchmark TODOs. Not yet included: - Transactional Firecracker dirty-epoch commit/abort. - Atomic and durable mem_image.json publication. - A dedicated lineage/lease object for artifact ownership. - Parent advancement for fork captures. - Chain compaction or garbage-collection policy. - Completed performance benchmarks. - Changes to the public API or OverlayBD format. ## Design and behavior changes launch parent | v current_memory_parent | +-- capture S1 succeeds locally | | | v | current_memory_parent = S1/mem_image.json | +-- capture S2 inherits S1 layers launch remains immutable and records where the Firecracker process started. current_memory_parent is initialized from it and advances independently. The update currently occurs after pause_to_dir produces the local snapshot configuration and before the VM resumes. Repository publication is not part of this local parent transition. ## Compatibility and operations - Public API or generated protocol: No changes. - Configuration or defaults: No changes. - Snapshot manifest, artifact layout, or storage format: No format changes; existing ordered OverlayBD lower layers are reused. - Upgrade and rollback: Existing snapshots remain readable. This Draft does not yet define mixed-version dirty-epoch behavior. - Host requirements, permissions, ports, or dependencies: No changes. ## Validation - [ ] make fmt - [ ] make clippy - [ ] make test-unit - [ ] Relevant Rust integration tests - [ ] make -C services test (required when services/ changes) - [ ] Generated clients/server regenerated with the documented make target - [x] Documentation updated - [ ] Benchmarks or performance comparison completed Commands and results: cargo fmt --all -- --check Passed. git diff --check Passed. cargo test -p agentenv --lib current_memory_parent_advances_without_changing_launch_provenance Did not compile on macOS because the Linux-only io-uring dependency requires MAP_POPULATE, MADV_DONTFORK, cpu_set_t, and io_uring syscall constants. The test executable was not run. Skipped checks and reasons: Full Rust and integration tests require a supported Linux host. No service or generated-code checks are applicable. Performance measurements remain TODO. The planned benchmark repeatedly rewrites a fixed hot-page set and records: - Per-capture pause latency. - Emitted memory-layer bytes. - Chain depth. - Artifact-retention cost. - Behavior before and after the compaction threshold. - Results after 10 and 100 captures. ## Risks and reviewer notes This is intentionally an incomplete Draft. The current parent update is not yet coupled to an atomic image-config commit or transactional dirty-epoch acknowledgement. Fork captures also need to use the same transition. Before merge, artifact ownership should move into an explicit lineage/lease abstraction so every referenced local layer remains pinned. Linear chains retain obsolete versions of repeatedly written hot pages. Compaction policy and benchmark results are required before enabling this behavior in production. The main review points are: - Whether current lineage should live directly in FirecrackerSandbox. - Whether local durable capture is the correct epoch commit boundary. - How fork captures participate in the same lineage. - What chain-depth and compaction limits are acceptable. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-29 14:20:56 +00:00

bsakaguchi referenced this issue

2026-10-29 14:21:25 +00:00

[PR #198] [MERGED] fix: support dnf and yum in install script #246