Files
ComfyUI_frontend/docs
snomiao 87b3f13f87 feat(agent): experimental in-browser ComfyAI agent
Squashes 60 incremental agent commits into one cohesive change after
prior history accumulated 119 unrelated upstream commits during a
bad rebase. Working tree restored from sno-agent@aec9f9380; new deps
added freshly (no stale lockfile entries); changeTracker API call
sites updated for current sno-frontend-preview (captureCanvasState →
checkState).

What's in this commit:

UI: src/agent/ui/{AgentRoot,AgentFab,XtermPanel,AgentSettings,
useXtermReadline}.vue
- Floating ComfyAI button + draggable xterm panel toggled via 'c'
  keybind; auto-focuses the terminal helper textarea on open.
- xterm-driven readline with Tab completion, Shift+Enter newline,
  Ctrl-A/E/U/K/L/C, history navigation, multiline buffers.
- Liquid-glass theme using Comfy design tokens.
- Settings panel auto-opens when no API key set; compact 3-field
  layout (API base / API key / model) + collapsed advanced section.

Session loop: src/agent/llm/session.ts, composables/useAgentSession.ts
- Vercel AI SDK streamText with run_shell as the only tool.
- IndexedDB-persisted message history (300-message cap), replays
  on reopen with a 'previous session' divider.
- Programmatic guardrails: PROMISSORY_PATTERN auto-continue,
  silent-fail auto-continue, fragile-shell-idiom blocklist
  (Layer 1), definitive-claim verifier registry (Layer 2 — orphans,
  missing-models, queue-state, pre-refusal, punt-to-user).
- Configurable baseURL for OpenRouter / local LLM proxies; default
  gpt-5.4 via OpenAI.

Shell runtime: src/agent/shell/
- POSIX-ish parser (shell-quote based), AsyncIterable pipes,
  redirection, &&/||/; sequencing.
- VFS: in-memory /tmp + UserdataVFS-backed /workflows.
- Coreutils + Comfy.* command dispatch + run-js fallback.

Commands: src/agent/shell/commands/
- comfy/comfyNs: registered command discovery + namespace dispatch.
- workflow: save-as / new-workflow / rename-workflow / clear-workflow
  --force / set-subgraph-{desc,aliases} (modal-bypass equivalents).
- nodeOps: node-search, add-node (smart placement), connect (auto-
  layout on link), disconnect, remove-node, layout, align-nodes,
  distribute-nodes, select, get-widget, toggle-panel.
- graph/state/execution/sweep: introspection + queue + sweep helpers.
- templates / images / install / validate / see: template loading,
  output→input copy, Manager model install, Gemini canvas vision.

Tests: browser_tests/tests/agentTerminal.spec.ts, plus
src/agent/**/*.test.ts unit tests.

Docs: docs/adr/0009-frontend-only-agent-and-local-agent-bridge.md
records the architectural choice + rejected alternatives.

i18n: minimal en additions in src/locales/en/main.json for
agent.{fab,input,panel,settings} namespaces.

Deps added: @ai-sdk/openai, ai, zod, shell-quote, idb-keyval,
@xterm/xterm, @xterm/addon-fit, es-toolkit, @types/shell-quote.

PR #11547 — experimental, draft, expect breaking changes.
Preview: https://pr-11547.comfy-ui.pages.dev/
2026-05-02 17:39:21 +09:00
..