[PR #73] [CLOSED] [3/4] feat(template): host-side COPY plan (archive rewrite to guest paths) #158

New issue

Closed

opened 2026-10-29 14:20:26 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:20:26 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/73
Author: @JoyboyBrian
Created: 7/29/2026
Status: ❌ Closed

Base: main ← Head: e2b-build/05-copy-plan

📝 Commits (3)

81a8edb feat(snapshot): template build-context archive store (posixfs + oss)
d11d80a feat(api): E2B build-context upload endpoints and [template_build] config
a5377b6 feat(template): host-side COPY plan (archive rewrite to guest paths)

📊 Changes

24 files changed (+3378 additions, -2 deletions)

View changed files

📝 config/default.toml (+24 -0)
➕ src/api/build_files.rs (+287 -0)
📝 src/api/generated/src/apis/templates.rs (+29 -0)
📝 src/api/generated/src/models.rs (+158 -0)
📝 src/api/generated/src/server/mod.rs (+172 -0)
📝 src/api/impls/mod.rs (+4 -0)
📝 src/api/impls/template.rs (+95 -0)
📝 src/api/mod.rs (+1 -0)
📝 src/api/openapi.yml (+48 -0)
📝 src/api/server.rs (+3 -2)
📝 src/cfg.rs (+223 -0)
📝 src/snapshot/image_export/service.rs (+2 -0)
📝 src/snapshot/manager.rs (+8 -0)
➕ src/snapshot/repository/backends/oss/build_files.rs (+169 -0)
📝 src/snapshot/repository/backends/oss/mod.rs (+1 -0)
📝 src/snapshot/repository/backends/oss/repository.rs (+12 -0)
📝 src/snapshot/repository/backends/posixfs/backend.rs (+12 -0)
➕ src/snapshot/repository/backends/posixfs/build_files.rs (+726 -0)
📝 src/snapshot/repository/backends/posixfs/mod.rs (+2 -0)
➕ src/snapshot/repository/build_files.rs (+197 -0)

...and 4 more files

📄 Description

What

Add a host-side COPY planner that reads an uploaded build-context archive and emits a bounded rewritten tar whose entries are mapped to their final guest paths.

This is layer 3 of the active COPY/ADD stack, replayed onto current main at 00ba6cb without #70.

Layer	PR	Branch	Scope
1/4	#71	`e2b-build/03-build-file-store`	Build-context archive store
2/4	#72	`e2b-build/04-upload-api`	Upload API and configuration
3/4	#73	`e2b-build/05-copy-plan`	Host-side COPY planning
4/4	#74	`e2b-build/06-copy-exec`	COPY/ADD execution, E2E, and documentation

Why

COPY/ADD requires Docker-compatible source matching and destination mapping without allowing an untrusted context archive to escape the intended guest paths. Keeping this logic host-side makes validation deterministic before anything is uploaded to the build sandbox.

Related: #28. The original PR remains the full concept/reference discussion.

Scope and non-goals

This layer adds only src/template/copy_plan.rs and its private module declaration. It has no runtime caller until #74, so the temporary dead_code allowance is removed by the next layer. Storage, HTTP upload, guest transfer/extraction, and build-spec wiring are outside this PR.

Exact layer diff: e2b-build/04-upload-api...e2b-build/05-copy-plan

Design

The archive is opened once and processed in two bounded passes.
The first pass normalizes/indexes paths and computes source-to-destination mappings.
The second pass streams selected entries into a rewritten tar with final absolute paths and requested ownership/mode metadata.
Traversal, unsafe links, unsupported entry types, invalid paths, excessive entry counts, and decompressed-byte budget violations are rejected.
Gzip contexts, glob expansion, file/directory destinations, and metadata overrides are covered by unit tests.

Compatibility and operations

Pure internal logic with no public API or configuration change.
No snapshot/storage format change.
No new crate or host dependency.
No behavior is enabled until #74 wires the planner into template execution.

Validation

The rewritten stack was validated on Linux against the #74 tip:

cargo fmt --all -- --check
cargo test --locked -p agentenv --lib template::
# 68 passed, 0 failed
cargo clippy --locked -p agentenv --lib -- -D warnings

This layer's focused tests cover path resolution, globs, destination semantics, metadata, gzip, traversal/link safety, unsupported types, and entry/byte budgets. GitHub CI validates this cumulative branch independently.

Reviewer notes

Archive path rewriting and decompression bounds are the primary security review surfaces. #70 is not required to merge this stack.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/73 **Author:** [@JoyboyBrian](https://github.com/JoyboyBrian) **Created:** 7/29/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `e2b-build/05-copy-plan` --- ### 📝 Commits (3) - [`81a8edb`](https://github.com/kvcache-ai/AgentENV/commit/81a8edbae406df6d93a21063322370cf1bffd109) feat(snapshot): template build-context archive store (posixfs + oss) - [`d11d80a`](https://github.com/kvcache-ai/AgentENV/commit/d11d80a782eae96a71302fe2a699d8741dfec682) feat(api): E2B build-context upload endpoints and [template_build] config - [`a5377b6`](https://github.com/kvcache-ai/AgentENV/commit/a5377b62355b5c60fd77090c2bf138909f1372b4) feat(template): host-side COPY plan (archive rewrite to guest paths) ### 📊 Changes **24 files changed** (+3378 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `config/default.toml` (+24 -0) ➕ `src/api/build_files.rs` (+287 -0) 📝 `src/api/generated/src/apis/templates.rs` (+29 -0) 📝 `src/api/generated/src/models.rs` (+158 -0) 📝 `src/api/generated/src/server/mod.rs` (+172 -0) 📝 `src/api/impls/mod.rs` (+4 -0) 📝 `src/api/impls/template.rs` (+95 -0) 📝 `src/api/mod.rs` (+1 -0) 📝 `src/api/openapi.yml` (+48 -0) 📝 `src/api/server.rs` (+3 -2) 📝 `src/cfg.rs` (+223 -0) 📝 `src/snapshot/image_export/service.rs` (+2 -0) 📝 `src/snapshot/manager.rs` (+8 -0) ➕ `src/snapshot/repository/backends/oss/build_files.rs` (+169 -0) 📝 `src/snapshot/repository/backends/oss/mod.rs` (+1 -0) 📝 `src/snapshot/repository/backends/oss/repository.rs` (+12 -0) 📝 `src/snapshot/repository/backends/posixfs/backend.rs` (+12 -0) ➕ `src/snapshot/repository/backends/posixfs/build_files.rs` (+726 -0) 📝 `src/snapshot/repository/backends/posixfs/mod.rs` (+2 -0) ➕ `src/snapshot/repository/build_files.rs` (+197 -0) _...and 4 more files_ </details> ### 📄 Description ## What Add a host-side COPY planner that reads an uploaded build-context archive and emits a bounded rewritten tar whose entries are mapped to their final guest paths. This is layer 3 of the active COPY/ADD stack, replayed onto current main at `00ba6cb` without #70. | Layer | PR | Branch | Scope | |---|---|---|---| | 1/4 | #71 | `e2b-build/03-build-file-store` | Build-context archive store | | 2/4 | #72 | `e2b-build/04-upload-api` | Upload API and configuration | | **3/4** | **#73** | **`e2b-build/05-copy-plan`** | **Host-side COPY planning** | | 4/4 | #74 | `e2b-build/06-copy-exec` | COPY/ADD execution, E2E, and documentation | ## Why COPY/ADD requires Docker-compatible source matching and destination mapping without allowing an untrusted context archive to escape the intended guest paths. Keeping this logic host-side makes validation deterministic before anything is uploaded to the build sandbox. Related: #28. The original PR remains the full concept/reference discussion. ## Scope and non-goals This layer adds only `src/template/copy_plan.rs` and its private module declaration. It has no runtime caller until #74, so the temporary `dead_code` allowance is removed by the next layer. Storage, HTTP upload, guest transfer/extraction, and build-spec wiring are outside this PR. Exact layer diff: [e2b-build/04-upload-api...e2b-build/05-copy-plan](https://github.com/JoyboyBrian/AgentENV/compare/e2b-build/04-upload-api...e2b-build/05-copy-plan) ## Design - The archive is opened once and processed in two bounded passes. - The first pass normalizes/indexes paths and computes source-to-destination mappings. - The second pass streams selected entries into a rewritten tar with final absolute paths and requested ownership/mode metadata. - Traversal, unsafe links, unsupported entry types, invalid paths, excessive entry counts, and decompressed-byte budget violations are rejected. - Gzip contexts, glob expansion, file/directory destinations, and metadata overrides are covered by unit tests. ## Compatibility and operations - Pure internal logic with no public API or configuration change. - No snapshot/storage format change. - No new crate or host dependency. - No behavior is enabled until #74 wires the planner into template execution. ## Validation The rewritten stack was validated on Linux against the #74 tip: ```text cargo fmt --all -- --check cargo test --locked -p agentenv --lib template:: # 68 passed, 0 failed cargo clippy --locked -p agentenv --lib -- -D warnings ``` This layer's focused tests cover path resolution, globs, destination semantics, metadata, gzip, traversal/link safety, unsupported types, and entry/byte budgets. GitHub CI validates this cumulative branch independently. ## Reviewer notes Archive path rewriting and decompression bounds are the primary security review surfaces. #70 is not required to merge this stack. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:20:26 +00:00