kvcache-ai/firecracker

Fork 0

mirror of https://github.com/kvcache-ai/firecracker.git synced 2026-11-05 01:17:43 +00:00

[PR #30] feat: add explicit dirty memory clearing API #30

New issue

Open

opened 2026-10-29 14:30:16 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:30:16 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/firecracker/pull/30
Author: @Hygge-Gezelligheid
Created: 10/28/2026
Status: 🔄 Open

Base: v1.15.1-patch-nestedvirt ← Head: feat/clear-dirty-memory

📝 Commits (3)

b52c645 feat: add explicit dirty memory clearing API [skip ci]
e9181ca test: streamline CLEAR coverage and validate native Diff [skip ci]
bc2020b test: verify the shared dirty baseline across snapshot consumers [skip ci]

📊 Changes

9 files changed (+923 additions, -2 deletions)

View changed files

📝 CHANGELOG.md (+7 -0)
📝 docs/snapshotting/snapshot-support.md (+57 -0)
📝 src/firecracker/src/api_server/parsed_request.rs (+62 -1)
📝 src/firecracker/src/api_server/request/snapshot.rs (+37 -0)
📝 src/firecracker/swagger/firecracker.yaml (+36 -0)
📝 src/vmm/src/rpc_interface.rs (+300 -1)
📝 tests/framework/http_api.py (+3 -0)
➕ tests/host_tools/dirty_memory_helper.c (+66 -0)
➕ tests/integration_tests/functional/test_dirty_memory_clear.py (+355 -0)

📄 Description

External memory exporters can query dirty ranges, but GET preserves their logical dirty records. Add PUT /vm/dirty-memory-ranges/clear to retire those records after committing a snapshot and establish the baseline for subsequent incremental exports.

Behavior

Accept {} and return 204 No Content; require a started, paused VM with dirty tracking enabled.
Read and clear plugged KVM slots, reset Firecracker's bitmap after all slot reads succeed, and conservatively mark activated MMIO/PCI virtio queue pages dirty.
Preserve guest bytes and existing GET behavior. Keep one pause window across state/device capture, memory export, successful snapshot commit and CLEAR; coordinate asynchronous and external memory writers.
Reject invalid requests and unmet preconditions before clearing. A KVM slot failure returns an error without rollback; the caller must treat the instance as unreliable and recover from its committed snapshot in a replacement instance.
All native snapshots and external exports share one global dirty baseline. Given native Full S0, externally committed changes A producing S1, CLEAR, then changes B, the next native Diff must be applied to S1. Applying it directly to S0 omits A. CLEAR has no consumer-specific history, GET token or generation ID.

The cross-consumer test verifies the omitted A against the wrong S0 base, restores the standalone native Diff from the correct S1 base, then performs another CLEAR and native Diff after restore. No dirty GET occurs between the new guest writes and either native Diff. Other coverage includes request validation, preconditions, queue pages, zero writes, no-GET clearing, and asynchronous block activity during state capture.

Scope and baseline

This PR contains three CLEAR commits across nine files, based directly on v1.15.1-patch-nestedvirt at c7e1cceb6bf643f239ddac804a535ceb8ac93cb1. It can be reviewed and merged independently of #27. Its required GET, memory-export and dirty-bitmap interfaces already exist in this target branch.

Complete CLEAR change

Validation

Fresh validation of bc2020bd372666919a554ab9954affc5cc80acfe on native x86_64 KVM with Rust 1.93.0:

Musl release builds of Firecracker, jailer and snapshot-editor passed.
Nine CLEAR unit tests and one GET regression passed.
All 12 CLEAR functional cases passed. Shared-baseline coverage uses Linux 5.10.245 and 6.1.155 guests with PCI enabled and disabled.
All 11 selected existing snapshot regressions passed: dirty pages after Full, Diff after a failed snapshot, resume, load-failure handling and negative snapshot creation.
All four Python formatting/import-order/pylint and Swagger checks passed. The changed snapshot documentation passed mdformat, and git diff --check passed.
git range-diff confirms that the three CLEAR patches are preserved; the only patch-context changes are in the changelog after removing the previous baseline's release section.

Earlier aarch64 runs and the broader workspace, fault-injection, performance, hotplug and security runs were performed on the previous stacked history. They are historical evidence, not fresh results for this rebased head. The full workspace/security matrix and aarch64 were not rerun for this history separation.

Known limitations

Local fault injection previously reproduced the existing GET failure problem: a later slot failure loses KVM-only bits already consumed from preceding slots, and GET retry cannot recover them. This PR does not change that production path. Controlled async tests do not establish quiescence for arbitrary external memory writers.

Earlier broad validation was not fully green: it reported an existing rand unsoundness advisory, host/guest CPU vulnerability checks, ARM host/toolchain limitations, and unavailable ARM guest/cross-kernel test assets. The focused validation above does not resolve those findings or establish a complete security certification. Select the required minor API version before release.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/firecracker/pull/30 **Author:** [@Hygge-Gezelligheid](https://github.com/Hygge-Gezelligheid) **Created:** 10/28/2026 **Status:** 🔄 Open **Base:** `v1.15.1-patch-nestedvirt` ← **Head:** `feat/clear-dirty-memory` --- ### 📝 Commits (3) - [`b52c645`](https://github.com/kvcache-ai/firecracker/commit/b52c645cd5f3bafc5abb353ce3b132378ab60511) feat: add explicit dirty memory clearing API [skip ci] - [`e9181ca`](https://github.com/kvcache-ai/firecracker/commit/e9181ca6700ce3b9a074f2966b1a795c4ec41569) test: streamline CLEAR coverage and validate native Diff [skip ci] - [`bc2020b`](https://github.com/kvcache-ai/firecracker/commit/bc2020bd372666919a554ab9954affc5cc80acfe) test: verify the shared dirty baseline across snapshot consumers [skip ci] ### 📊 Changes **9 files changed** (+923 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `CHANGELOG.md` (+7 -0) 📝 `docs/snapshotting/snapshot-support.md` (+57 -0) 📝 `src/firecracker/src/api_server/parsed_request.rs` (+62 -1) 📝 `src/firecracker/src/api_server/request/snapshot.rs` (+37 -0) 📝 `src/firecracker/swagger/firecracker.yaml` (+36 -0) 📝 `src/vmm/src/rpc_interface.rs` (+300 -1) 📝 `tests/framework/http_api.py` (+3 -0) ➕ `tests/host_tools/dirty_memory_helper.c` (+66 -0) ➕ `tests/integration_tests/functional/test_dirty_memory_clear.py` (+355 -0) </details> ### 📄 Description External memory exporters can query dirty ranges, but GET preserves their logical dirty records. Add `PUT /vm/dirty-memory-ranges/clear` to retire those records after committing a snapshot and establish the baseline for subsequent incremental exports. ## Behavior - Accept `{}` and return `204 No Content`; require a started, paused VM with dirty tracking enabled. - Read and clear plugged KVM slots, reset Firecracker's bitmap after all slot reads succeed, and conservatively mark activated MMIO/PCI virtio queue pages dirty. - Preserve guest bytes and existing GET behavior. Keep one pause window across state/device capture, memory export, successful snapshot commit and CLEAR; coordinate asynchronous and external memory writers. - Reject invalid requests and unmet preconditions before clearing. A KVM slot failure returns an error without rollback; the caller must treat the instance as unreliable and recover from its committed snapshot in a replacement instance. - All native snapshots and external exports share one global dirty baseline. Given native Full S0, externally committed changes A producing S1, CLEAR, then changes B, the next native Diff must be applied to S1. Applying it directly to S0 omits A. CLEAR has no consumer-specific history, GET token or generation ID. The cross-consumer test verifies the omitted A against the wrong S0 base, restores the standalone native Diff from the correct S1 base, then performs another CLEAR and native Diff after restore. No dirty GET occurs between the new guest writes and either native Diff. Other coverage includes request validation, preconditions, queue pages, zero writes, no-GET clearing, and asynchronous block activity during state capture. ## Scope and baseline This PR contains three CLEAR commits across nine files, based directly on `v1.15.1-patch-nestedvirt` at `c7e1cceb6bf643f239ddac804a535ceb8ac93cb1`. It can be reviewed and merged independently of #27. Its required GET, memory-export and dirty-bitmap interfaces already exist in this target branch. [Complete CLEAR change](https://github.com/Hygge-Gezelligheid/firecracker/compare/c7e1cceb6bf643f239ddac804a535ceb8ac93cb1...bc2020bd372666919a554ab9954affc5cc80acfe) ## Validation Fresh validation of `bc2020bd372666919a554ab9954affc5cc80acfe` on native x86_64 KVM with Rust 1.93.0: - Musl release builds of Firecracker, jailer and snapshot-editor passed. - Nine CLEAR unit tests and one GET regression passed. - All 12 CLEAR functional cases passed. Shared-baseline coverage uses Linux 5.10.245 and 6.1.155 guests with PCI enabled and disabled. - All 11 selected existing snapshot regressions passed: dirty pages after Full, Diff after a failed snapshot, resume, load-failure handling and negative snapshot creation. - All four Python formatting/import-order/pylint and Swagger checks passed. The changed snapshot documentation passed mdformat, and `git diff --check` passed. - `git range-diff` confirms that the three CLEAR patches are preserved; the only patch-context changes are in the changelog after removing the previous baseline's release section. Earlier aarch64 runs and the broader workspace, fault-injection, performance, hotplug and security runs were performed on the previous stacked history. They are historical evidence, not fresh results for this rebased head. The full workspace/security matrix and aarch64 were not rerun for this history separation. ## Known limitations Local fault injection previously reproduced the existing GET failure problem: a later slot failure loses KVM-only bits already consumed from preceding slots, and GET retry cannot recover them. This PR does not change that production path. Controlled async tests do not establish quiescence for arbitrary external memory writers. Earlier broad validation was not fully green: it reported an existing rand unsoundness advisory, host/guest CPU vulnerability checks, ARM host/toolchain limitations, and unavailable ARM guest/cross-kernel test assets. The focused validation above does not resolve those findings or establish a complete security certification. Select the required minor API version before release. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi added the

pull-request

label

2026-10-29 14:30:16 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/firecracker#30

No description provided.

Rows
Columns