diff --git a/docs/architecture/graph-architecture-evolution.md b/docs/architecture/graph-architecture-evolution.md new file mode 100644 index 000000000..f28b0bbe5 --- /dev/null +++ b/docs/architecture/graph-architecture-evolution.md @@ -0,0 +1,177 @@ +# Graph Architecture Evolution + +A visual journey through the architectural transformation of ComfyUI's graph system. + +--- + +## Slide 1: Traditional LiteGraph Architecture + +``` +┌─────────────────────────────────────┐ +│ LiteGraph Node │ +│ ┌─────────────────────────────┐ │ +│ │ • Data (inputs/outputs) │ │ +│ │ • UI (position, size) │ │ +│ │ • Rendering (draw methods) │ │ +│ │ • Interaction (mouse) │ │ +│ │ • Business Logic │ │ +│ └─────────────────────────────┘ │ +│ │ +│ Everything tightly coupled in │ +│ a single monolithic structure │ +└─────────────────────────────────────┘ +``` + +**Problem**: All concerns mixed together - data, UI, rendering, and interaction are inseparable. + +--- + +## Slide 2: Separation of Concerns + +``` +┌─────────────────────┐ ┌──────────────────────┐ +│ Graph Data Model │ │ Layout Tree │ +├─────────────────────┤ ├──────────────────────┤ +│ • Node connections │ │ • Node positions │ +│ • Input/output data │ │ • Node sizes │ +│ • Execution state │ │ • Z-index/stacking │ +│ • Business logic │ │ • Visibility states │ +│ │ │ • Bounds/spatial │ +│ Pure data structure │ │ Pure UI structure │ +│ No UI concepts │ │ No business logic │ +└─────────────────────┘ └──────────────────────┘ +``` + +**Benefit**: Clean separation - graph logic vs presentation concerns. + +--- + +## Slide 3: Multiple Renderer Support + +``` + Layout Tree + │ + ┌────────────────┼────────────────┐ + ▼ ▼ ▼ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Canvas Render│ │ Vue Render │ │ Three.js 3D │ +├──────────────┤ ├──────────────┤ ├──────────────┤ +│ │ │ │ │ │ +│ [Canvas] │ │ │ │ [WebGL] │ +│ │ │ │ │ │ +└──────────────┘ └──────────────┘ └──────────────┘ + + ┌────────────────┼────────────────┐ + ▼ ▼ ▼ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Native iOS │ │Native Android│ │ Terminal │ +├──────────────┤ ├──────────────┤ ├──────────────┤ +│ UIKit │ │ Compose │ │ ASCII │ +└──────────────┘ └──────────────┘ └──────────────┘ +``` + +**Power**: Same layout tree, infinite rendering possibilities. + +--- + +## Slide 4: Alternative UI Paradigms + +``` + Graph Data Model + │ + ├─────── With Layout Tree ─────► Traditional Node UI + │ + └─────── Without Layout ───────┐ + ▼ + ┌──────────────────┐ + │ Form-Based UI │ + ├──────────────────┤ + │ ┌──────────────┐ │ + │ │ Input Fields │ │ + │ ├──────────────┤ │ + │ │ Sliders │ │ + │ ├──────────────┤ │ + │ │ Buttons │ │ + │ └──────────────┘ │ + │ │ + │ Like Gradio/A1111│ + └──────────────────┘ +``` + +**Flexibility**: Renderers can interpret the graph data model however they want. + +--- + +## Slide 5: Service Architecture + +``` +┌─────────────────────┐ ┌──────────────────────┐ +│ Graph Mutations │ │ Layout Mutations │ +│ Service │ │ Service │ +├─────────────────────┤ ├──────────────────────┤ +│ • addNode() │ │ • moveNode() │ +│ • connectNodes() │ │ • resizeNode() │ +│ • updateNodeData() │ │ • bringToFront() │ +│ • executeGraph() │ │ • setVisibility() │ +└──────┬──────────────┘ └──────┬───────────────┘ + │ │ + ▼ ▼ +┌─────────────────────┐ ┌──────────────────────┐ +│ Graph Gateway │ │ Layout Gateway │ +│ (Interface) │ │ (Interface) │ +└─────────────────────┘ └──────────────────────┘ +``` + +**Clean APIs**: Well-defined interfaces for all operations. + +--- + +## Slide 6: Deployment Flexibility + +``` +┌────────────────────────────────────────────────┐ +│ Graph Data Model │ +│ Layout System │ +│ Service Layer │ +└─────────────┬───────────┬───────────┬──────────┘ + │ │ │ + Local │ Cloud │ Native │ Hybrid + ▼ ▼ ▼ ▼ + ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ + │Browser │ │ AWS │ │ iOS │ │Electron│ + │ │ │Lambda │ │ App │ │ + │ + │ Yjs │ │ + │ │ + │ │ Rust │ + │ CRDT │ │GraphQL │ │ Swift │ │Backend │ + └────────┘ └────────┘ └────────┘ └────────┘ + + Because everything is behind interfaces, easily switch implementations without major development effort: + • Move graph execution to GPU cluster + • Run layout calculations in WebAssembly + • Store state in any database + • Sync across any network protocol +``` + +**Ultimate Flexibility**: Plug and play any component, deploy anywhere. + +--- + +## Summary + +By separating concerns and defining clear interfaces: + +1. **Data Model** → Pure business logic, no UI +2. **Layout Tree** → Pure spatial data, no logic +3. **Renderers** → Consume what they need +4. **Services** → Clean mutation APIs +5. **Gateways** → Swappable implementations, automatically map across differing API versions or schemas + +This architecture enables: +- Multiple simultaneous renderers +- Alternative UI paradigms +- Cloud/edge/native deployment +- Real-time collaboration +- Time-travel debugging +- Performance optimization per layer +- Better performance for state observers and undo/redo + +The key insight: **When you separate concerns properly, everything becomes possible.** \ No newline at end of file diff --git a/src/services/README.md b/src/services/README.md index 00061b771..afefba70d 100644 --- a/src/services/README.md +++ b/src/services/README.md @@ -1,155 +1,277 @@ -# Reactive Layout Services +# Services -This directory contains the core implementations of the reactive layout system that bridges Vue node interactions with LiteGraph. +This directory contains the service layer for the ComfyUI frontend application. Services encapsulate application logic and functionality into organized, reusable modules. + +## Table of Contents + +- [Overview](#overview) +- [Service Architecture](#service-architecture) +- [Core Services](#core-services) +- [Service Development Guidelines](#service-development-guidelines) +- [Common Design Patterns](#common-design-patterns) + +## Overview + +Services in ComfyUI provide organized modules that implement the application's functionality and logic. They handle operations such as API communication, workflow management, user settings, and other essential features. + +The term "business logic" in this context refers to the code that implements the core functionality and behavior of the application - the rules, processes, and operations that make ComfyUI work as expected, separate from the UI display code. + +Services help organize related functionality into cohesive units, making the codebase more maintainable and testable. By centralizing related operations in services, the application achieves better separation of concerns, with UI components focusing on presentation and services handling functional operations. ## Service Architecture -```mermaid -graph LR - subgraph "Services" - RLT[ReactiveLayoutTree
- Position/Bounds State
- Selection State] - RHT[ReactiveHitTester
- Spatial Queries
- QuadTree Integration] - end +The service layer in ComfyUI follows these architectural principles: - subgraph "Renderers" - Canvas[Canvas Renderer
(LiteGraph)] - Vue[Vue Renderer
(DOM Nodes)] - end +1. **Domain-driven**: Each service focuses on a specific domain of the application +2. **Stateless when possible**: Services generally avoid maintaining internal state +3. **Reusable**: Services can be used across multiple components +4. **Testable**: Services are designed for easy unit testing +5. **Isolated**: Services have clear boundaries and dependencies - subgraph "Spatial Index" - QT[QuadTree
Spatial Index] - end +While services can interact with both UI components and stores (centralized state), they primarily focus on implementing functionality rather than managing state. The following diagram illustrates how services fit into the application architecture: - Canvas -->|Write| RLT - Vue -->|Write| RLT - RLT -->|Reactive Updates| Canvas - RLT -->|Reactive Updates| Vue - - RHT -->|Query| QT - RLT -->|Sync Bounds| RHT - RHT -->|Hit Testing| Vue - - -## ReactiveLayoutTree Implementation - -```mermaid -classDiagram - class ReactiveLayoutTree { - -_nodePositions: Ref~Map~ - -_nodeBounds: Ref~Map~ - -_selectedNodes: Ref~Set~ - +nodePositions: ComputedRef~Map~ - +nodeBounds: ComputedRef~Map~ - +selectedNodes: Ref~Set~ - +updateNodePosition(nodeId, position) - +updateNodeBounds(nodeId, bounds) - +selectNodes(nodeIds, addToSelection) - +clearSelection() - } - - class customRef { - <> - +track() - +trigger() - } - - ReactiveLayoutTree --> customRef : uses for shared write access +``` +┌─────────────────────────────────────────────────────────┐ +│ UI Components │ +└────────────────────────────┬────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────┐ +│ Composables │ +└────────────────────────────┬────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────┐ +│ Services │ +│ │ +│ (Application Functionality) │ +└────────────────────────────┬────────────────────────────┘ + │ + ┌───────────┴───────────┐ + ▼ ▼ +┌───────────────────────────┐ ┌─────────────────────────┐ +│ Stores │ │ External APIs │ +│ (Centralized State) │ │ │ +└───────────────────────────┘ └─────────────────────────┘ ``` -### Key Features -- Uses Vue's `customRef` to allow both renderers to write -- Provides reactive computed properties for automatic updates -- Maintains immutable update pattern (creates new Maps on change) -- Supports both single and bulk updates +## Core Services -## ReactiveHitTester Implementation +The following table lists ALL services in the system as of 2025-01-30: -```mermaid -flowchart TB - subgraph "Hit Testing Flow" - Query[Spatial Query] - QT[QuadTree Index] - Candidates[Candidate Nodes] - Precise[Precise Bounds Check] - Result[Hit Test Result] - end +### Main Services - Query -->|Viewport Bounds| QT - QT -->|Fast Filter| Candidates - Candidates -->|Intersection Test| Precise - Precise --> Result +| Service | Description | Category | +|---------|-------------|----------| +| autoQueueService.ts | Manages automatic queue execution | Execution | +| colorPaletteService.ts | Handles color palette management and customization | UI | +| comfyManagerService.ts | Manages ComfyUI application packages and updates | Manager | +| comfyRegistryService.ts | Handles registration and discovery of ComfyUI extensions | Registry | +| dialogService.ts | Provides dialog and modal management | UI | +| extensionService.ts | Manages extension registration and lifecycle | Extensions | +| keybindingService.ts | Handles keyboard shortcuts and keybindings | Input | +| litegraphService.ts | Provides utilities for working with the LiteGraph library | Graph | +| load3dService.ts | Manages 3D model loading and visualization | 3D | +| nodeHelpService.ts | Provides node documentation and help | Nodes | +| nodeOrganizationService.ts | Handles node organization and categorization | Nodes | +| nodeSearchService.ts | Implements node search functionality | Search | +| releaseService.ts | Manages application release information and updates | System | +| subgraphService.ts | Handles subgraph operations and navigation | Graph | +| workflowService.ts | Handles workflow operations (save, load, execute) | Workflows | - subgraph "Reactive Queries" - RP[Reactive Point Query] - RB[Reactive Bounds Query] - Auto[Auto-update on Layout Change] - end +### Gateway Services +Located in `services/gateway/`: - RP --> Query - RB --> Query - Auto -.->|Triggers| RP - Auto -.->|Triggers| RB -``` +| Service | Description | +|---------|-------------| +| registrySearchGateway.ts | Gateway for registry search operations | -### Performance Optimizations -- Integrates with existing QuadTree spatial indexing -- Two-phase hit testing: spatial index filter + precise bounds check -- Reactive queries use Vue's computed for efficient caching -- Direct queries available for immediate results during interactions +### Provider Services +Located in `services/providers/`: -## Data Synchronization +| Service | Description | +|---------|-------------| +| algoliaSearchProvider.ts | Implements search functionality using Algolia | +| registrySearchProvider.ts | Provides registry search capabilities | -```mermaid -sequenceDiagram - participant LG as LiteGraph - participant LT as LayoutTree - participant HT as HitTester - participant SI as Spatial Index - participant VN as Vue Node +## Service Development Guidelines - Note over LG,VN: Initial Sync - LG->>LT: Bulk position update - LT->>HT: Bounds changed (reactive) - HT->>SI: Batch update spatial index +In ComfyUI, services can be implemented using two approaches: - Note over LG,VN: Vue Node Drag - VN->>VN: CSS transform (visual) - VN->>LT: updateNodePosition (on drag end) - LT->>LG: Position changed (reactive watch) - LT->>HT: Bounds changed (reactive) - HT->>SI: Update node in index - LG->>LG: Redraw canvas +### 1. Class-based Services - Note over LG,VN: Canvas Drag - LG->>LG: Update node.pos - LG->>LT: Sync position (RAF) - LT->>HT: Bounds changed (reactive) - HT->>SI: Update node in index - LT->>VN: Position changed (reactive) -``` - -## Usage Example +For complex services with state management and multiple methods, class-based services are used: ```typescript -// In Vue component -const { layoutTree, hitTester } = useReactiveLayout() +export class NodeSearchService { + // Service state + private readonly nodeFuseSearch: FuseSearch + private readonly filters: Record> -// Initialize layout tree sync -const { initializeSync } = useLiteGraphSync() -initializeSync() + constructor(data: ComfyNodeDefImpl[]) { + // Initialize state + this.nodeFuseSearch = new FuseSearch(data, { /* options */ }) + + // Setup filters + this.filters = { + inputType: new FuseFilter(/* options */), + category: new FuseFilter(/* options */) + } + } -// In Vue node component -const { - isDragging, - startDrag, - handleDrag, - endDrag, - dragStyle -} = useVueNodeInteraction(nodeId) + public searchNode(query: string, filters: FuseFilterWithValue[] = []): ComfyNodeDefImpl[] { + // Implementation + return results + } +} +``` -// Reactive position tracking -const nodePos = hitTester.getNodePosition(nodeId) -watch(nodePos, (newPos) => { - console.log('Node moved to:', newPos) -}) -``` \ No newline at end of file +### 2. Composable-style Services + +For simpler services or those that need to integrate with Vue's reactivity system, we prefer using composable-style services: + +```typescript +export function useNodeSearchService(initialData: ComfyNodeDefImpl[]) { + // State (reactive if needed) + const data = ref(initialData) + + // Search functionality + function searchNodes(query: string) { + // Implementation + return results + } + + // Additional methods + function refreshData(newData: ComfyNodeDefImpl[]) { + data.value = newData + } + + // Return public API + return { + searchNodes, + refreshData + } +} +``` + +When deciding between these approaches, consider: + +1. **Stateful vs. Stateless**: For stateful services, classes often provide clearer encapsulation +2. **Reactivity needs**: If the service needs to be reactive, composable-style services integrate better with Vue's reactivity system +3. **Complexity**: For complex services with many methods and internal state, classes can provide better organization +4. **Testing**: Both approaches can be tested effectively, but composables may be simpler to test with Vue Test Utils + +### Service Template + +Here's a template for creating a new composable-style service: + +```typescript +/** + * Service for managing [domain/functionality] + */ +export function useExampleService() { + // Private state/functionality + const cache = new Map() + + /** + * Description of what this method does + * @param param1 Description of parameter + * @returns Description of return value + */ + async function performOperation(param1: string) { + try { + // Implementation + return result + } catch (error) { + // Error handling + console.error(`Operation failed: ${error.message}`) + throw error + } + } + + // Return public API + return { + performOperation + } +} +``` + +## Common Design Patterns + +Services in ComfyUI frequently use the following design patterns: + +### Caching and Request Deduplication + +```typescript +export function useCachedService() { + const cache = new Map() + const pendingRequests = new Map() + + async function fetchData(key: string) { + // Check cache first + if (cache.has(key)) return cache.get(key) + + // Check if request is already in progress + if (pendingRequests.has(key)) { + return pendingRequests.get(key) + } + + // Perform new request + const requestPromise = fetch(`/api/${key}`) + .then(response => response.json()) + .then(data => { + cache.set(key, data) + pendingRequests.delete(key) + return data + }) + + pendingRequests.set(key, requestPromise) + return requestPromise + } + + return { fetchData } +} +``` + +### Factory Pattern + +```typescript +export function useNodeFactory() { + function createNode(type: string, config: Record) { + // Create node based on type and configuration + switch (type) { + case 'basic': + return { /* basic node implementation */ } + case 'complex': + return { /* complex node implementation */ } + default: + throw new Error(`Unknown node type: ${type}`) + } + } + + return { createNode } +} +``` + +### Facade Pattern + +```typescript +export function useWorkflowService( + apiService, + graphService, + storageService +) { + // Provides a simple interface to complex subsystems + async function saveWorkflow(name: string) { + const graphData = graphService.serializeGraph() + const storagePath = await storageService.getPath(name) + return apiService.saveData(storagePath, graphData) + } + + return { saveWorkflow } +} +``` + +For more detailed information about the service layer pattern and its applications, refer to: +- [Service Layer Pattern](https://en.wikipedia.org/wiki/Service_layer_pattern) +- [Service-Orientation](https://en.wikipedia.org/wiki/Service-orientation) \ No newline at end of file diff --git a/src/types/README.md b/src/types/README.md deleted file mode 100644 index 04bccb127..000000000 --- a/src/types/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Reactive Layout Types - -This directory contains type definitions for the reactive layout system that enables Vue nodes to handle their own interactions while staying synchronized with LiteGraph. - -## Architecture Overview - -```mermaid -graph TB - subgraph "Type Definitions" - Point[Point: x, y] - Size[Size: width, height] - Bounds[Bounds: x, y, width, height] - SlotRef[SlotRef: nodeId, slotIndex, isOutput] - end - - subgraph "Core Interfaces" - LayoutTree[LayoutTree
- nodePositions
- nodeBounds
- selectedNodes] - HitTester[HitTester
- getNodeAt
- getNodesInBounds] - GraphMutationService[GraphMutationService
- moveNode
- selectNode
- connectNodes] - InteractionState[InteractionState
- dragState
- selectionState] - end - - subgraph "Renderer Interface" - GraphRenderer[GraphRenderer
- setLayoutTree
- render
- mount/unmount] - end - - Point --> Bounds - Size --> Bounds - Bounds --> LayoutTree - Bounds --> HitTester - Point --> GraphMutationService - SlotRef --> GraphMutationService - LayoutTree --> GraphRenderer - HitTester --> GraphRenderer - - -## Data Flow During Interactions - -```mermaid -sequenceDiagram - participant User - participant VueNode - participant LayoutTree - participant LiteGraph - - User->>VueNode: Drag Start - VueNode->>VueNode: Apply CSS Transform - Note over VueNode: Visual feedback only - - User->>VueNode: Drag Move - VueNode->>VueNode: Update CSS Transform - Note over VueNode: Smooth dragging - - User->>VueNode: Drag End - VueNode->>LayoutTree: updateNodePosition() - LayoutTree->>LiteGraph: Reactive sync - LiteGraph->>LiteGraph: Update canvas -``` - -## Key Interfaces - -### LayoutTree -- Manages spatial/visual information reactively -- Provides reactive getters for positions, bounds, and selection -- Allows both Canvas and Vue renderers to update during transition - -### HitTester -- Provides spatial queries (find nodes at point, in bounds) -- Offers both reactive (auto-updating) and direct queries -- Integrates with QuadTree spatial indexing for performance - -### GraphMutationService -- Future API for all graph data changes -- Separates data mutations from layout updates -- Will be the single point of access for graph modifications - -### InteractionState -- Tracks user interactions reactively -- Manages drag and selection state -- Provides actions for state transitions - \ No newline at end of file