merge main into rh-test

This commit is contained in:
bymyself
2025-09-28 15:33:29 -07:00
parent 1c0f151d02
commit ff0c15b119
1317 changed files with 85439 additions and 18373 deletions

293
docs/SETTINGS.md Normal file
View File

@@ -0,0 +1,293 @@
# Settings System
## Overview
ComfyUI frontend uses a comprehensive settings system for user preferences with support for dynamic defaults, version-based rollouts, and environment-aware configuration.
### Settings Architecture
- Settings are defined as `SettingParams` in `src/constants/coreSettings.ts`
- Registered at app startup, loaded/saved via `useSettingStore` (Pinia)
- Persisted per user via backend `/settings` endpoint
- If a value hasn't been set by the user, the store returns the computed default
```typescript
// From src/stores/settingStore.ts:105-122
function getDefaultValue<K extends keyof Settings>(
key: K
): Settings[K] | undefined {
const param = getSettingById(key)
if (param === undefined) return
const versionedDefault = getVersionedDefaultValue(key, param)
if (versionedDefault) {
return versionedDefault
}
return typeof param.defaultValue === 'function'
? param.defaultValue()
: param.defaultValue
}
```
### Settings Registration Process
Settings are registered after server values are loaded:
```typescript
// From src/components/graph/GraphCanvas.vue:311-315
CORE_SETTINGS.forEach((setting) => {
settingStore.addSetting(setting)
})
await newUserService().initializeIfNewUser(settingStore)
```
## Dynamic and Environment-Based Defaults
### Computed Defaults
You can compute defaults dynamically using function defaults that access runtime context:
```typescript
// From src/constants/coreSettings.ts:94-101
{
id: 'Comfy.Sidebar.Size',
// Default to small if the window is less than 1536px(2xl) wide
defaultValue: () => (window.innerWidth < 1536 ? 'small' : 'normal')
}
```
```typescript
// From src/constants/coreSettings.ts:306
{
id: 'Comfy.Locale',
defaultValue: () => navigator.language.split('-')[0] || 'en'
}
```
### Version-Based Defaults
You can vary defaults by installed frontend version using `defaultsByInstallVersion`:
```typescript
// From src/stores/settingStore.ts:129-150
function getVersionedDefaultValue<K extends keyof Settings, TValue = Settings[K]>(
key: K,
param: SettingParams<TValue> | undefined
): TValue | null {
const defaultsByInstallVersion = param?.defaultsByInstallVersion
if (defaultsByInstallVersion && key !== 'Comfy.InstalledVersion') {
const installedVersion = get('Comfy.InstalledVersion')
if (installedVersion) {
const sortedVersions = Object.keys(defaultsByInstallVersion).sort(
(a, b) => compareVersions(b, a)
)
for (const version of sortedVersions) {
if (!isSemVer(version)) continue
if (compareVersions(installedVersion, version) >= 0) {
const versionedDefault = defaultsByInstallVersion[version]
return typeof versionedDefault === 'function'
? versionedDefault()
: versionedDefault
}
}
}
}
return null
}
```
Example versioned defaults from codebase:
```typescript
// From src/constants/coreSettings.ts:38-40
{
id: 'Comfy.Graph.LinkReleaseAction',
defaultValue: LinkReleaseTriggerAction.CONTEXT_MENU,
defaultsByInstallVersion: {
'1.24.1': LinkReleaseTriggerAction.SEARCH_BOX
}
}
// Another versioned default example
{
id: 'Comfy.Graph.LinkReleaseAction.Shift',
defaultValue: LinkReleaseTriggerAction.SEARCH_BOX,
defaultsByInstallVersion: {
'1.24.1': LinkReleaseTriggerAction.CONTEXT_MENU
}
}
```
### Real Examples from Codebase
Here are actual settings showing different patterns:
```typescript
// Number setting with validation
{
id: 'LiteGraph.Node.TooltipDelay',
name: 'Tooltip Delay',
type: 'number',
attrs: {
min: 100,
max: 3000,
step: 50
},
defaultValue: 500,
versionAdded: '1.9.0'
}
// Hidden system setting for tracking
{
id: 'Comfy.InstalledVersion',
name: 'The frontend version that was running when the user first installed ComfyUI',
type: 'hidden',
defaultValue: null,
versionAdded: '1.24.0'
}
// Slider with complex tooltip
{
id: 'LiteGraph.Canvas.LowQualityRenderingZoomThreshold',
name: 'Low quality rendering zoom threshold',
tooltip: 'Zoom level threshold for performance mode. Lower values (0.1) = quality at all zoom levels. Higher values (1.0) = performance mode even when zoomed in.',
type: 'slider',
attrs: {
min: 0.1,
max: 1.0,
step: 0.05
},
defaultValue: 0.5
}
```
### New User Version Capture
The initial installed version is captured for new users to ensure versioned defaults remain stable:
```typescript
// From src/services/newUserService.ts:49-53
await settingStore.set(
'Comfy.InstalledVersion',
__COMFYUI_FRONTEND_VERSION__
)
```
## Practical Patterns for Environment-Based Defaults
### Dynamic Default Patterns
```typescript
// Device-based default
{
id: 'Comfy.Example.MobileDefault',
type: 'boolean',
defaultValue: () => /Mobile/i.test(navigator.userAgent)
}
// Environment-based default
{
id: 'Comfy.Example.DevMode',
type: 'boolean',
defaultValue: () => import.meta.env.DEV
}
// Window size based
{
id: 'Comfy.Example.CompactUI',
type: 'boolean',
defaultValue: () => window.innerWidth < 1024
}
```
### Version-Based Rollout Pattern
```typescript
{
id: 'Comfy.Example.NewFeature',
type: 'combo',
options: ['legacy', 'enhanced'],
defaultValue: 'legacy',
defaultsByInstallVersion: {
'1.25.0': 'enhanced'
}
}
```
## Settings Persistence and Access
### API Interaction
Values are stored per user via the backend. The store writes through API and falls back to defaults when not set:
```typescript
// From src/stores/settingStore.ts:73-75
onChange(settingsById.value[key], newValue, oldValue)
settingValues.value[key] = newValue
await api.storeSetting(key, newValue)
```
### Usage in Components
```typescript
const settingStore = useSettingStore()
// Get setting value (returns computed default if not set by user)
const value = settingStore.get('Comfy.SomeSetting')
// Update setting value
await settingStore.set('Comfy.SomeSetting', newValue)
```
## Advanced Settings Features
### Migration and Backward Compatibility
Settings support migration from deprecated values:
```typescript
// From src/stores/settingStore.ts:68-69, 172-175
const newValue = tryMigrateDeprecatedValue(
settingsById.value[key],
clonedValue
)
// Migration happens during addSetting for existing values:
if (settingValues.value[setting.id] !== undefined) {
settingValues.value[setting.id] = tryMigrateDeprecatedValue(
setting,
settingValues.value[setting.id]
)
}
```
### onChange Callbacks
Settings can define onChange callbacks that receive the setting definition, new value, and old value:
```typescript
// From src/stores/settingStore.ts:73, 177
onChange(settingsById.value[key], newValue, oldValue) // During set()
onChange(setting, get(setting.id), undefined) // During addSetting()
```
### Settings UI and Categories
Settings are automatically grouped for UI based on their `category` or derived from `id`:
```typescript
{
id: 'Comfy.Sidebar.Size',
category: ['Appearance', 'Sidebar', 'Size'],
// UI will group this under Appearance > Sidebar > Size
}
```
## Related Documentation
- Feature flag system: `docs/FEATURE_FLAGS.md`
- Settings schema for backend: `src/schemas/apiSchema.ts` (zSettings)
- Server configuration (separate from user settings): `src/constants/serverConfig.ts`
## Summary
- **Settings**: User preferences with dynamic/versioned defaults, persisted per user
- **Environment Defaults**: Use function defaults to read runtime context (window, navigator, env)
- **Version Rollouts**: Use `defaultsByInstallVersion` for gradual feature releases
- **API Interaction**: Settings persist to `/settings` endpoint via `storeSetting()`

View File

@@ -0,0 +1,82 @@
# Settings and Feature Flags Sequence Diagram
This diagram shows the flow of settings initialization, default resolution, persistence, and feature flags exchange.
This diagram accurately reflects the actual implementation in the ComfyUI frontend codebase.
```mermaid
sequenceDiagram
participant User as User
participant Vue as Vue Component
participant Store as SettingStore (Pinia)
participant API as ComfyApi (WebSocket/REST)
participant Backend as Backend
participant NewUserSvc as NewUserService
Note over Vue,Store: App startup (GraphCanvas.vue)
Vue->>Store: loadSettingValues()
Store->>API: getSettings()
API->>Backend: GET /settings
Backend-->>API: settings map (per-user)
API-->>Store: settings map
Store-->>Vue: loaded
Vue->>Store: register CORE_SETTINGS (addSetting for each)
loop For each setting registration
Store->>Store: tryMigrateDeprecatedValue(existing value)
Store->>Store: onChange(setting, currentValue, undefined)
end
Note over Vue,NewUserSvc: New user detection
Vue->>NewUserSvc: initializeIfNewUser(settingStore)
NewUserSvc->>NewUserSvc: checkIsNewUser(settingStore)
alt New user detected
NewUserSvc->>Store: set("Comfy.InstalledVersion", __COMFYUI_FRONTEND_VERSION__)
Store->>Store: tryMigrateDeprecatedValue(newValue)
Store->>Store: onChange(setting, newValue, oldValue)
Store->>API: storeSetting(key, newValue)
API->>Backend: POST /settings/{id}
else Existing user
Note over NewUserSvc: Skip setting installed version
end
Note over Vue,Store: Component reads a setting
Vue->>Store: get(key)
Store->>Store: exists(key)?
alt User value exists
Store-->>Vue: return stored user value
else Not set by user
Store->>Store: getVersionedDefaultValue(key)
alt Versioned default matched (defaultsByInstallVersion)
Store-->>Vue: return versioned default
else No version match
Store->>Store: evaluate defaultValue (function or constant)
Note over Store: defaultValue can use window size,<br/>locale, env, etc.
Store-->>Vue: return computed default
end
end
Note over User,Store: User updates a setting
User->>Vue: changes setting in UI
Vue->>Store: set(key, newValue)
Store->>Store: tryMigrateDeprecatedValue(newValue)
Store->>Store: check if newValue === oldValue (early return if same)
Store->>Store: onChange(setting, newValue, oldValue)
Store->>Store: update settingValues[key]
Store->>API: storeSetting(key, newValue)
API->>Backend: POST /settings/{id}
Backend-->>API: 200 OK
API-->>Store: ack
Note over API,Backend: Feature Flags WebSocket Exchange
API->>Backend: WS connect
API->>Backend: send { type: "feature_flags", data: clientFeatureFlags.json }
Backend-->>API: WS send { type: "feature_flags", data: server flags }
API->>API: store serverFeatureFlags = data
Note over Vue,API: Feature flag consumption in UI/logic
Vue->>API: serverSupportsFeature(name)
API-->>Vue: boolean (true only if flag === true)
Vue->>API: getServerFeature(name, default)
API-->>Vue: value or default
```

View File

@@ -0,0 +1,156 @@
# 3. Centralized Layout Management with CRDT
Date: 2025-08-27
## Status
Proposed
## Context
ComfyUI's node graph editor currently suffers from fundamental architectural limitations around spatial data management that prevent us from achieving key product goals.
### Current Architecture Problems
The existing system allows each node to directly mutate its position within LiteGraph's canvas renderer. This creates several critical issues:
1. **Performance Bottlenecks**: UI updates require full graph traversals to detect position changes. Large workflows (100+ nodes) can create bottlenecks during interactions due to this O(n) polling approach.
2. **Position Conflicts**: Multiple systems (LiteGraph canvas, DOMwidgets.ts overlays) currently compete to control node positions. Future Vue widget overlays will compound this maintenance burden.
3. **No Collaboration Foundation**: Direct position mutations make concurrent editing impossible—there's no mechanism to merge conflicting position updates from multiple users.
4. **Renderer Lock-in**: Spatial data is tightly coupled to LiteGraph's canvas implementation, preventing alternative rendering approaches (WebGL, DOM, other libraries, hybrid approaches).
5. **Inefficient Change Detection**: While LiteGraph provides some events, many operations require polling via changeTracker.ts. The current undo/redo system performs expensive diffs on every interaction rather than using reactive push/pull signals, creating performance bottlenecks and blocking efficient animations and viewport culling.
This represents a fundamental architectural limitation: diff-based systems scale O(n) with graph complexity (traverse entire structure to detect changes), while signal-based reactive systems scale O(1) with actual changes (data mutations automatically notify subscribers). Modern frameworks (Vue 3, Angular signals, SolidJS) have moved to reactive approaches for precisely this performance reason.
### Business Context
- Performance issues emerge with workflow complexity (100+ nodes)
- The AI workflow community increasingly expects collaborative features (similar to Figma, Miro)
- Accessibility requirements will necessitate DOM-based rendering options
- Technical debt compounds with each new spatial feature
This decision builds on [ADR-0001 (Merge LiteGraph)](0001-merge-litegraph-into-frontend.md), which enables the architectural restructuring proposed here.
## Decision
We will implement a centralized layout management system using CRDT (Conflict-free Replicated Data Types) with command pattern architecture to separate spatial data from rendering behavior.
### Centralized State Management Foundation
This solution applies proven centralized state management patterns:
- **Centralized Store**: All spatial data (position, size, bounds, transform) managed in a single CRDT-backed store
- **Command Interface**: All mutations flow through explicit commands rather than direct property access
- **Observer Pattern**: Independent systems (rendering, interaction, layout) subscribe to state changes
- **Domain Separation**: Layout logic completely separated from rendering and UI concerns
This provides single source of truth, predictable state updates, and natural system decoupling—solving our core architectural problems.
### Core Architecture
1. **Centralized Layout Store**: A Yjs CRDT maintains all spatial data in a single authoritative store:
```typescript
// Instead of: node.position = {x, y}
layoutStore.moveNode(nodeId, {x, y})
```
2. **Command Pattern**: All spatial mutations flow through explicit commands:
```
User Input → Commands → Layout Store → Observer Notifications → Renderers
```
3. **Observer-Based Systems**: Multiple independent systems subscribe to layout changes:
- **Rendering Systems**: LiteGraph canvas, WebGL, DOM accessibility renderers
- **Interaction Systems**: Drag handlers, selection, hover states
- **Layout Systems**: Auto-layout, alignment, distribution
- **Animation Systems**: Smooth transitions, physics simulations
4. **Reactive Updates**: Store changes propagate through observers, eliminating polling and enabling efficient system coordination.
### Implementation Strategy
**Phase 1: Parallel System**
- Build CRDT layout store alongside existing system
- Layout store initially mirrors LiteGraph changes via observers
- Gradually migrate user interactions to use command interface
- Maintain full backward compatibility
**Phase 2: Inversion of Control**
- CRDT store becomes single source of truth
- LiteGraph receives position updates via reactive subscriptions
- Enable alternative renderers and advanced features
### Why Centralized State + CRDT?
This combination provides both architectural and technical benefits:
**Centralized State Benefits:**
- **Single Source of Truth**: All layout data managed in one place, eliminating conflicts
- **System Decoupling**: Rendering, interaction, and layout systems operate independently
- **Predictable Updates**: Clear data flow makes debugging and testing easier
- **Extensibility**: Easy to add new layout behaviors without modifying existing systems
**CRDT Benefits:**
- **Conflict Resolution**: Automatic merging eliminates position conflicts between systems
- **Collaboration-Ready**: Built-in support for multi-user editing
- **Eventual Consistency**: Guaranteed convergence to same state across all clients
**Yjs-Specific Benefits:**
- **Event-Driven**: Native observer pattern removes need for polling
- **Selective Updates**: Only changed nodes trigger system updates
- **Fine-Grained Changes**: Efficient delta synchronization
## Consequences
### Positive
- **Eliminates Polling**: Observer pattern removes O(n) graph traversals, improving performance
- **System Modularity**: Independent systems can be developed, tested, and optimized separately
- **Renderer Flexibility**: Easy to add WebGL, DOM accessibility, or hybrid rendering systems
- **Rich Interactions**: Command pattern enables robust undo/redo, macros, and interaction history
- **Collaboration-Ready**: CRDT foundation enables real-time multi-user editing
- **Conflict Resolution**: Eliminates position "snap-back" behavior between competing systems
- **Better Developer Experience**: Clear separation of concerns and predictable data flow patterns
### Negative
- **Learning Curve**: Team must understand CRDT concepts and centralized state management
- **Migration Complexity**: Gradual migration of existing direct property access requires careful coordination
- **Memory Overhead**: Yjs library (~30KB) plus operation history storage
- **CRDT Performance**: CRDTs have computational overhead compared to direct property access
- **Increased Abstraction**: Additional layer between user interactions and visual updates
### Risk Mitigations
- Provide comprehensive migration documentation and examples
- Build compatibility layer for gradual, low-risk migration
- Implement operation history pruning for long-running sessions
- Phase implementation to validate approach before full migration
## Notes
This centralized state + CRDT architecture follows patterns from modern collaborative applications:
**Centralized State Management**: Similar to Redux/Vuex patterns in complex web applications, but with CRDT backing for collaboration. This provides predictable state updates while enabling real-time multi-user features.
**CRDT in Collaboration**: Tools like Figma, Linear, and Notion use similar approaches for real-time collaboration, demonstrating the effectiveness of separating authoritative data from presentation logic.
**Future Capabilities**: This foundation enables advanced features that would be difficult with the current architecture:
- Macro recording and workflow automation
- Programmatic layout optimization and constraints
- API-driven workflow construction
- Multiple simultaneous renderers (canvas + accessibility DOM)
- Real-time collaborative editing
- Advanced spatial features (physics, animations, auto-layout)
The architecture provides immediate single-user benefits while creating infrastructure for collaborative and advanced spatial features.
## References
- [Yjs Documentation](https://docs.yjs.dev/)
- [CRDTs: The Hard Parts](https://martin.kleppmann.com/2020/07/06/crdt-hard-parts-hydra.html) by Martin Kleppmann
- [Figma's Multiplayer Technology](https://www.figma.com/blog/how-figmas-multiplayer-technology-works/)

View File

@@ -0,0 +1,62 @@
# 4. Fork PrimeVue UI Library
Date: 2025-08-27
## Status
Rejected
## Context
ComfyUI's frontend requires modifications to PrimeVue components that cannot be achieved through the library's customization APIs. Two specific technical incompatibilities have been identified with the transform-based canvas architecture:
**Screen Coordinate Hit-Testing Conflicts:**
- PrimeVue components use `getBoundingClientRect()` for screen coordinate calculations that don't account for CSS transforms
- The Slider component directly uses raw `pageX/pageY` coordinates ([lines 102-103](https://github.com/primefaces/primevue/blob/master/packages/primevue/src/slider/Slider.vue#L102-L103)) without transform-aware positioning
- This breaks interaction in transformed coordinate spaces where screen coordinates don't match logical element positions
**Virtual Canvas Scroll Interference:**
- LiteGraph's infinite canvas uses scroll coordinates semantically for graph navigation via the `DragAndScale` coordinate system
- PrimeVue overlay components automatically trigger `scrollIntoView` behavior which interferes with this virtual positioning
- This issue is documented in [PrimeVue discussion #4270](https://github.com/orgs/primefaces/discussions/4270) where the feature request was made to disable this behavior
**Historical Overlay Issues:**
- Previous z-index positioning conflicts required manual workarounds (commit `6d4eafb0`) where PrimeVue Dialog components needed `autoZIndex: false` and custom mask styling, later resolved by removing PrimeVue's automatic z-index management entirely
**Minimal Update Overhead:**
- Analysis of git history shows only 2 PrimeVue version updates in 2+ years, indicating that upstream sync overhead is negligible for this project
**Future Interaction System Requirements:**
- The ongoing canvas architecture evolution will require more granular control over component interaction and event handling as the transform-based system matures
- Predictable need for additional component modifications beyond current identified issues
## Decision
We will **NOT** fork PrimeVue. After evaluation, forking was determined to be unnecessarily complex and costly.
**Rationale for Rejection:**
- **Significant Implementation Complexity**: PrimeVue is structured as a monorepo ([primefaces/primevue](https://github.com/primefaces/primevue)) with significant code in a separate monorepo ([PrimeUIX](https://github.com/primefaces/primeuix)). Forking would require importing both repositories whole and selectively pruning or exempting components from our workspace tooling, adding substantial complexity.
- **Alternative Solutions Available**: The modifications we identified (e.g., scroll interference issues, coordinate system conflicts) have less costly solutions that don't require maintaining a full fork. For example, coordinate issues could be addressed through event interception and synthetic event creation with scaled values.
- **Maintenance Burden**: Ongoing maintenance and upgrades would be very painful, requiring manual conflict resolution and keeping pace with upstream changes across multiple repositories.
- **Limited Tooling Support**: There isn't adequate tooling that provides the granularity needed to cleanly manage a PrimeVue fork within our existing infrastructure.
## Consequences
### Alternative Approach
- **Use PrimeVue as External Dependency**: Continue using PrimeVue as a standard npm dependency
- **Targeted Workarounds**: Implement specific solutions for identified issues (coordinate system conflicts, scroll interference) without forking the entire library
- **Selective Component Replacement**: Use libraries like shadcn/ui to replace specific problematic PrimeVue components and adjust them to match our design system
- **Upstream Engagement**: Continue engaging with PrimeVue community for feature requests and bug reports
- **Maintain Flexibility**: Preserve ability to upgrade PrimeVue versions without fork maintenance overhead
## Notes
- Technical issues documented in the Context section remain valid concerns
- Solutions will be pursued through targeted fixes rather than wholesale forking
- Future re-evaluation possible if PrimeVue's architecture significantly changes or if alternative tooling becomes available
- This decision prioritizes maintainability and development velocity over maximum customization control

View File

@@ -11,7 +11,9 @@ An Architecture Decision Record captures an important architectural decision mad
| ADR | Title | Status | Date |
|-----|-------|--------|------|
| [0001](0001-merge-litegraph-into-frontend.md) | Merge LiteGraph.js into ComfyUI Frontend | Accepted | 2025-08-05 |
| [0002](0002-monorepo-conversion.md) | Restructure as a Monorepo | Proposed | 2025-08-25 |
| [0002](0002-monorepo-conversion.md) | Restructure as a Monorepo | Accepted | 2025-08-25 |
| [0003](0003-crdt-based-layout-system.md) | Centralized Layout Management with CRDT | Proposed | 2025-08-27 |
| [0004](0004-fork-primevue-ui-library.md) | Fork PrimeVue UI Library | Rejected | 2025-08-27 |
## Creating a New ADR
@@ -77,4 +79,4 @@ Optional section for additional information, references, or clarifications.
## Further Reading
- [Documenting Architecture Decisions](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) by Michael Nygard
- [Architecture Decision Records](https://adr.github.io/) - Collection of ADR resources
- [Architecture Decision Records](https://adr.github.io/) - Collection of ADR resources

View File

@@ -110,7 +110,7 @@ pnpm build
For faster iteration during development, use watch mode:
```bash
npx vite build --watch
pnpm exec vite build --watch
```
Note: Watch mode provides faster rebuilds than full builds, but still no hot reload