ADD steps and document E2B builds #159

New issue

Closed

opened 2026-10-29 14:20:26 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:20:26 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/74
Author: @JoyboyBrian
Created: 7/29/2026
Status: ❌ Closed

Base: main ← Head: e2b-build/06-copy-exec

📝 Commits (4)

81a8edb feat(snapshot): template build-context archive store (posixfs + oss)
d11d80a feat(api): E2B build-context upload endpoints and [template_build] config
a5377b6 feat(template): host-side COPY plan (archive rewrite to guest paths)
730cc51 feat(template): execute COPY/ADD steps and document E2B builds

📊 Changes

35 files changed (+4283 additions, -34 deletions)

View changed files

📝 config/default.toml (+24 -0)
📝 docs/src/integration/e2b.md (+28 -0)
📝 scripts/tests/e2e/e2b_python_sdk_compat.py (+23 -4)
📝 scripts/tests/e2e/suites/09_e2b_compat.sh (+2 -2)
➕ src/api/build_files.rs (+287 -0)
📝 src/api/generated/src/apis/templates.rs (+29 -0)
📝 src/api/generated/src/models.rs (+158 -0)
📝 src/api/generated/src/server/mod.rs (+172 -0)
📝 src/api/impls/mod.rs (+4 -0)
📝 src/api/impls/template.rs (+95 -0)
📝 src/api/impls/template_helpers.rs (+185 -13)
📝 src/api/mod.rs (+1 -0)
📝 src/api/openapi.yml (+48 -0)
📝 src/api/server.rs (+3 -2)
📝 src/cfg.rs (+223 -0)
📝 src/sandbox/backend.rs (+14 -0)
📝 src/sandbox/envd.rs (+35 -0)
📝 src/sandbox/process.rs (+12 -0)
📝 src/snapshot/image_export/service.rs (+2 -0)
📝 src/snapshot/manager.rs (+8 -0)

...and 15 more files

📄 Description

What

Wire E2B COPY and ADD execution end to end:

accept and validate COPY/ADD build steps;
collect and materialize referenced context archives;
rewrite them through #73's host-side planner;
upload the rewritten tar to the build sandbox through envd;
resolve optional --chown, preserve requested modes, and extract once with tar;
enforce archive-count and aggregate-context budgets;
document E2B template builds and add a fresh-template SDK E2E covering both COPY and ADD.

This is the final layer of the active four-PR stack, replayed onto current main at 00ba6cb without #70.

Layer	PR	Branch	Scope
1/4	#71	`e2b-build/03-build-file-store`	Build-context archive store
2/4	#72	`e2b-build/04-upload-api`	Upload API and configuration
3/4	#73	`e2b-build/05-copy-plan`	Host-side COPY planning
4/4	#74	`e2b-build/06-copy-exec`	COPY/ADD execution, E2E, and documentation

Why

This completes the user-visible E2B template-build workflow on top of #71–#73. The official SDK can upload a local Dockerfile context, reference it by filesHash, and build a reusable AgentENV template containing the copied files.

Related: #28. The original PR remains the full concept/reference discussion.

Scope and non-goals

This layer contains the COPY/ADD build-spec mapping, archive materialization, guest upload/extraction, sandbox plumbing, documentation, and E2E coverage. It removes #73's temporary dead_code allowance.

It does not introduce or claim ownership of WORKDIR behavior. WORKDIR directory creation and relative resolution already exist on current main; the E2E uses that existing behavior only to verify copied files from a realistic Dockerfile.

Existing-alias rebuild remains unsupported. Use a new alias or delete the existing template before rebuilding; #70 is parked pending the alias refactor.

Exact layer diff: e2b-build/05-copy-plan...e2b-build/06-copy-exec

Design

The builder collects unique filesHash references, enforces count/aggregate size limits, and materializes archives through the repository store.
The executor resolves optional user/group names inside the build VM, rewrites each archive on the host, uploads it through envd, and extracts with ownership/mode preservation and --no-overwrite-dir.
The planner remains responsible for guest-path safety and decompression bounds.
Documentation explains SDK usage, upload URL deployment requirements, ownership behavior, destination limitations, and the temporary existing-alias restriction.

Compatibility and operations

E2B COPY/ADD step shapes now build instead of returning the prior unsupported-step response.
Consumes the [template_build] limits introduced by #72; no additional setting is added here.
Committed snapshot manifests remain unchanged.
Uses existing envd file upload and guest tar; no new host dependency or port.

Validation

cargo fmt --all -- --check
cargo test --locked -p agentenv --lib template::
# 68 passed, 0 failed
cargo clippy --locked -p agentenv --lib -- -D warnings
make agentenv-server
# regeneration produced no intended generated-code diff

Fresh-template Linux/KVM E2E on devmachine:

E2B Python SDK 2.37.0
Dockerfile: FROM + WORKDIR + COPY + ADD
template build: passed
sandbox create/list: passed
copied and added file contents: passed
pause/reconnect and command execution: passed

The E2E uses a newly generated alias and temporary local build context, so it exercises the supported path independently of the parked alias-rebuild work.

Reviewer notes

The main review surfaces are archive resource bounds, path safety at the planner boundary, --chown identity resolution, and extraction semantics. Merge after #71–#73; #70 is not required.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/74 **Author:** [@JoyboyBrian](https://github.com/JoyboyBrian) **Created:** 7/29/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `e2b-build/06-copy-exec` --- ### 📝 Commits (4) - [`81a8edb`](https://github.com/kvcache-ai/AgentENV/commit/81a8edbae406df6d93a21063322370cf1bffd109) feat(snapshot): template build-context archive store (posixfs + oss) - [`d11d80a`](https://github.com/kvcache-ai/AgentENV/commit/d11d80a782eae96a71302fe2a699d8741dfec682) feat(api): E2B build-context upload endpoints and [template_build] config - [`a5377b6`](https://github.com/kvcache-ai/AgentENV/commit/a5377b62355b5c60fd77090c2bf138909f1372b4) feat(template): host-side COPY plan (archive rewrite to guest paths) - [`730cc51`](https://github.com/kvcache-ai/AgentENV/commit/730cc5154fac5af9180b05ff75d195845911c7f3) feat(template): execute COPY/ADD steps and document E2B builds ### 📊 Changes **35 files changed** (+4283 additions, -34 deletions) <details> <summary>View changed files</summary> 📝 `config/default.toml` (+24 -0) 📝 `docs/src/integration/e2b.md` (+28 -0) 📝 `scripts/tests/e2e/e2b_python_sdk_compat.py` (+23 -4) 📝 `scripts/tests/e2e/suites/09_e2b_compat.sh` (+2 -2) ➕ `src/api/build_files.rs` (+287 -0) 📝 `src/api/generated/src/apis/templates.rs` (+29 -0) 📝 `src/api/generated/src/models.rs` (+158 -0) 📝 `src/api/generated/src/server/mod.rs` (+172 -0) 📝 `src/api/impls/mod.rs` (+4 -0) 📝 `src/api/impls/template.rs` (+95 -0) 📝 `src/api/impls/template_helpers.rs` (+185 -13) 📝 `src/api/mod.rs` (+1 -0) 📝 `src/api/openapi.yml` (+48 -0) 📝 `src/api/server.rs` (+3 -2) 📝 `src/cfg.rs` (+223 -0) 📝 `src/sandbox/backend.rs` (+14 -0) 📝 `src/sandbox/envd.rs` (+35 -0) 📝 `src/sandbox/process.rs` (+12 -0) 📝 `src/snapshot/image_export/service.rs` (+2 -0) 📝 `src/snapshot/manager.rs` (+8 -0) _...and 15 more files_ </details> ### 📄 Description ## What Wire E2B `COPY` and `ADD` execution end to end: - accept and validate COPY/ADD build steps; - collect and materialize referenced context archives; - rewrite them through #73's host-side planner; - upload the rewritten tar to the build sandbox through envd; - resolve optional `--chown`, preserve requested modes, and extract once with `tar`; - enforce archive-count and aggregate-context budgets; - document E2B template builds and add a fresh-template SDK E2E covering both COPY and ADD. This is the final layer of the active four-PR stack, replayed onto current main at `00ba6cb` without #70. | Layer | PR | Branch | Scope | |---|---|---|---| | 1/4 | #71 | `e2b-build/03-build-file-store` | Build-context archive store | | 2/4 | #72 | `e2b-build/04-upload-api` | Upload API and configuration | | 3/4 | #73 | `e2b-build/05-copy-plan` | Host-side COPY planning | | **4/4** | **#74** | **`e2b-build/06-copy-exec`** | **COPY/ADD execution, E2E, and documentation** | ## Why This completes the user-visible E2B template-build workflow on top of #71–#73. The official SDK can upload a local Dockerfile context, reference it by `filesHash`, and build a reusable AgentENV template containing the copied files. Related: #28. The original PR remains the full concept/reference discussion. ## Scope and non-goals This layer contains the COPY/ADD build-spec mapping, archive materialization, guest upload/extraction, sandbox plumbing, documentation, and E2E coverage. It removes #73's temporary `dead_code` allowance. It does **not** introduce or claim ownership of WORKDIR behavior. WORKDIR directory creation and relative resolution already exist on current main; the E2E uses that existing behavior only to verify copied files from a realistic Dockerfile. Existing-alias rebuild remains unsupported. Use a new alias or delete the existing template before rebuilding; #70 is parked pending the alias refactor. Exact layer diff: [e2b-build/05-copy-plan...e2b-build/06-copy-exec](https://github.com/JoyboyBrian/AgentENV/compare/e2b-build/05-copy-plan...e2b-build/06-copy-exec) ## Design - The builder collects unique `filesHash` references, enforces count/aggregate size limits, and materializes archives through the repository store. - The executor resolves optional user/group names inside the build VM, rewrites each archive on the host, uploads it through envd, and extracts with ownership/mode preservation and `--no-overwrite-dir`. - The planner remains responsible for guest-path safety and decompression bounds. - Documentation explains SDK usage, upload URL deployment requirements, ownership behavior, destination limitations, and the temporary existing-alias restriction. ## Compatibility and operations - E2B COPY/ADD step shapes now build instead of returning the prior unsupported-step response. - Consumes the `[template_build]` limits introduced by #72; no additional setting is added here. - Committed snapshot manifests remain unchanged. - Uses existing envd file upload and guest `tar`; no new host dependency or port. ## Validation ```text cargo fmt --all -- --check cargo test --locked -p agentenv --lib template:: # 68 passed, 0 failed cargo clippy --locked -p agentenv --lib -- -D warnings make agentenv-server # regeneration produced no intended generated-code diff ``` Fresh-template Linux/KVM E2E on `devmachine`: ```text E2B Python SDK 2.37.0 Dockerfile: FROM + WORKDIR + COPY + ADD template build: passed sandbox create/list: passed copied and added file contents: passed pause/reconnect and command execution: passed ``` The E2E uses a newly generated alias and temporary local build context, so it exercises the supported path independently of the parked alias-rebuild work. ## Reviewer notes The main review surfaces are archive resource bounds, path safety at the planner boundary, `--chown` identity resolution, and extraction semantics. Merge after #71–#73; #70 is not required. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:20:26 +00:00