[PR #321] [MERGED] feat(templates): add E2B-compatible build status and logs #345

New issue

Closed

opened 2026-10-29 14:22:26 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 14:22:26 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/AgentENV/pull/321
Author: @zlzGithub-0801
Created: 9/26/2026
Status: ✅ Merged
Merged: 9/29/2026
Merged by: @yingdi-shan

Base: main ← Head: feat/template-status-logs

📝 Commits (1)

b5be3b9 feat(templates): add E2B-compatible build status and logs

📊 Changes

33 files changed (+2587 additions, -239 deletions)

View changed files

📝 crates/aenv/src/client/templates.rs (+11 -1)
📝 crates/aenv/src/commands/pull.rs (+22 -1)
📝 crates/aenv/src/commands/template.rs (+146 -46)
📝 crates/aenv/src/progress.rs (+18 -3)
📝 services/gateway/internal/server.go (+26 -6)
📝 services/gateway/internal/server_test.go (+45 -0)
📝 src/api/generated/src/apis/templates.rs (+31 -0)
📝 src/api/generated/src/models.rs (+266 -1)
📝 src/api/generated/src/server/mod.rs (+203 -4)
📝 src/api/impls/image_build.rs (+14 -3)
📝 src/api/impls/image_build/tests.rs (+255 -1)
📝 src/api/impls/image_build/worker.rs (+4 -1)
📝 src/api/impls/mod.rs (+2 -0)
📝 src/api/impls/template.rs (+386 -143)
📝 src/api/openapi.yml (+113 -1)
📝 src/image/build_history.proto (+29 -0)
📝 src/image/buildkit/history.rs (+247 -5)
📝 src/logging.rs (+13 -5)
📝 src/sandbox/process.rs (+35 -1)
📝 src/sandbox/ublk/device.rs (+1 -1)

...and 13 more files

📄 Description

What

Add E2B-compatible template build status and logs APIs, persistent build-log storage, and CLI build progress output.

Why

Template builds previously exposed limited status information and no structured logs. Users could not inspect build progress or diagnose failures through the API and CLI.

Scope and non-goals

This PR includes template build log collection, storage, API querying, internal build-node routing, and CLI output.

It does not change unrelated sandbox logging or retain completed build logs in memory.

Design and behavior changes

Log collection and storage

Template build logs are collected from three sources:

AgentENV tracing events within the template build span.
Sandbox command stdout/stderr through process output callbacks.
BuildKit vertex status, stdout/stderr, and warnings through its Status RPC.

Active logs are buffered in memory and periodically persisted to:

snapshots/build-logs/{buildID}.json

The final logs are flushed when the build finishes, after which the in-memory buffer is removed.

Status and logs APIs

The status endpoint returns build state together with paginated log entries. The logs endpoint supports cursor pagination, direction, level filtering, and log source selection.

temporary reads the active build’s in-memory buffer.
persistent reads the shared repository file.
The default source prefers temporary logs while the build is active and falls back to persistent logs after completion.

Build requests are routed internally to the build node without requiring clients to provide an additional routing header.

`aenv` CLI

aenv pull and aenv template watch now poll the status API with an incremental logsOffset and print new build logs while waiting.

The CLI displays info, warn, and error entries while filtering out debug entries. Detached pulls retain the existing behavior and return immediately without streaming logs.

Compatibility and operations

Public API or generated protocol: Adds template build status/logs definitions and the BuildKit Status RPC subset. Generated API code is updated.
Configuration or defaults: No new configuration.
Snapshot manifest, artifact layout, or storage format: Adds snapshots/build-logs/{buildID}.json.
Upgrade and rollback: Existing snapshots remain compatible. Rollback may leave unused build-log files.
Host requirements, permissions, ports, or dependencies: No new requirements.

Validation

make fmt
make clippy
make test-unit
Relevant Rust integration tests
make -C services test (required when services/ changes)
Generated clients/server regenerated with the documented make target
Documentation updated
Benchmarks or performance comparison completed

Risks and reviewer notes

BuildKit Status streams may replay events after reconnects, so vertex states and output entries are deduplicated.

Active builds use in-memory logs for low-latency reads. Completed builds rely on shared repository storage, so the final flush must complete before the buffer is removed.

The main review areas are:

Build-log lifecycle and persistence.
BuildKit replay and deduplication.
Internal routing to active build nodes.
Temporary and persistent API behavior.

Checklist

The PR contains one coherent change and no unrelated formatting or refactoring.
New behavior is covered by tests, or I explained why testing is impractical.
Logs and examples contain no credentials, tokens, or private registry information.
I did not manually edit generated code without updating its source and regenerating it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/AgentENV/pull/321 **Author:** [@zlzGithub-0801](https://github.com/zlzGithub-0801) **Created:** 9/26/2026 **Status:** ✅ Merged **Merged:** 9/29/2026 **Merged by:** [@yingdi-shan](https://github.com/yingdi-shan) **Base:** `main` ← **Head:** `feat/template-status-logs` --- ### 📝 Commits (1) - [`b5be3b9`](https://github.com/kvcache-ai/AgentENV/commit/b5be3b90c0a759cdada2834a8400687bc3afb2d9) feat(templates): add E2B-compatible build status and logs ### 📊 Changes **33 files changed** (+2587 additions, -239 deletions) <details> <summary>View changed files</summary> 📝 `crates/aenv/src/client/templates.rs` (+11 -1) 📝 `crates/aenv/src/commands/pull.rs` (+22 -1) 📝 `crates/aenv/src/commands/template.rs` (+146 -46) 📝 `crates/aenv/src/progress.rs` (+18 -3) 📝 `services/gateway/internal/server.go` (+26 -6) 📝 `services/gateway/internal/server_test.go` (+45 -0) 📝 `src/api/generated/src/apis/templates.rs` (+31 -0) 📝 `src/api/generated/src/models.rs` (+266 -1) 📝 `src/api/generated/src/server/mod.rs` (+203 -4) 📝 `src/api/impls/image_build.rs` (+14 -3) 📝 `src/api/impls/image_build/tests.rs` (+255 -1) 📝 `src/api/impls/image_build/worker.rs` (+4 -1) 📝 `src/api/impls/mod.rs` (+2 -0) 📝 `src/api/impls/template.rs` (+386 -143) 📝 `src/api/openapi.yml` (+113 -1) 📝 `src/image/build_history.proto` (+29 -0) 📝 `src/image/buildkit/history.rs` (+247 -5) 📝 `src/logging.rs` (+13 -5) 📝 `src/sandbox/process.rs` (+35 -1) 📝 `src/sandbox/ublk/device.rs` (+1 -1) _...and 13 more files_ </details> ### 📄 Description ## What Add E2B-compatible template build status and logs APIs, persistent build-log storage, and CLI build progress output. ## Why Template builds previously exposed limited status information and no structured logs. Users could not inspect build progress or diagnose failures through the API and CLI. ## Related issue #5 ## Scope and non-goals This PR includes template build log collection, storage, API querying, internal build-node routing, and CLI output. It does not change unrelated sandbox logging or retain completed build logs in memory. ## Design and behavior changes ### Log collection and storage Template build logs are collected from three sources: - AgentENV tracing events within the template build span. - Sandbox command stdout/stderr through process output callbacks. - BuildKit vertex status, stdout/stderr, and warnings through its Status RPC. Active logs are buffered in memory and periodically persisted to: ```text snapshots/build-logs/{buildID}.json ``` The final logs are flushed when the build finishes, after which the in-memory buffer is removed. ### Status and logs APIs The status endpoint returns build state together with paginated log entries. The logs endpoint supports cursor pagination, direction, level filtering, and log source selection. - `temporary` reads the active build’s in-memory buffer. - `persistent` reads the shared repository file. - The default source prefers temporary logs while the build is active and falls back to persistent logs after completion. Build requests are routed internally to the build node without requiring clients to provide an additional routing header. ### `aenv` CLI `aenv pull` and `aenv template watch` now poll the status API with an incremental `logsOffset` and print new build logs while waiting. The CLI displays `info`, `warn`, and `error` entries while filtering out `debug` entries. Detached pulls retain the existing behavior and return immediately without streaming logs. ## Compatibility and operations - Public API or generated protocol: Adds template build status/logs definitions and the BuildKit Status RPC subset. Generated API code is updated. - Configuration or defaults: No new configuration. - Snapshot manifest, artifact layout, or storage format: Adds `snapshots/build-logs/{buildID}.json`. - Upgrade and rollback: Existing snapshots remain compatible. Rollback may leave unused build-log files. - Host requirements, permissions, ports, or dependencies: No new requirements. ## Validation - [x] `make fmt` - [x] `make clippy` - [x] `make test-unit` - [x] Relevant Rust integration tests - [x] `make -C services test` (required when `services/` changes) - [x] Generated clients/server regenerated with the documented `make` target - [ ] Documentation updated - [ ] Benchmarks or performance comparison completed ## Risks and reviewer notes BuildKit Status streams may replay events after reconnects, so vertex states and output entries are deduplicated. Active builds use in-memory logs for low-latency reads. Completed builds rely on shared repository storage, so the final flush must complete before the buffer is removed. The main review areas are: - Build-log lifecycle and persistence. - BuildKit replay and deduplication. - Internal routing to active build nodes. - Temporary and persistent API behavior. ## Checklist - [x] The PR contains one coherent change and no unrelated formatting or refactoring. - [x] New behavior is covered by tests, or I explained why testing is impractical. - [x] Logs and examples contain no credentials, tokens, or private registry information. - [x] I did not manually edit generated code without updating its source and regenerating it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 14:22:26 +00:00