diff --git a/docs/ci/remove-dot-cache.md b/docs/ci/remove-dot-cache.md deleted file mode 100644 index a4bfa99a9..000000000 --- a/docs/ci/remove-dot-cache.md +++ /dev/null @@ -1,137 +0,0 @@ -# Performance Analysis: Removing `.cache` from GitHub Actions Workflows - -## Executive Summary - -**Recommendation:** Remove `.cache` directory caching from GitHub Actions workflows. - -**Impact:** Minimal to slightly positive performance impact (-1.2% total time on average), with improved maintainability. - -## Methodology - -Two test commits were created to measure performance impact: - -1. **Baseline (WITH .cache)**: [Commit 84b4956](https://github.com/Comfy-Org/ComfyUI_frontend/commit/84b49562c) - Temporarily restored `.cache` to all 8 affected workflows -2. **Comparison (WITHOUT .cache)**: [Commit 2af6f81](https://github.com/Comfy-Org/ComfyUI_frontend/commit/2af6f8137) - Reverted back to remove `.cache` - -All measurements were taken on standard GitHub Actions `ubuntu-latest` runners to ensure real-world data from production infrastructure. - -## Performance Results - -### Workflow Timing Comparison - -| Workflow | WITH .cache | WITHOUT .cache | Difference | % Change | -|----------|-------------|----------------|------------|----------| -| **CI: Tests Unit** | 173s (2.9m) | 174s (2.9m) | +1s | +0.6% | -| **CI: Lint Format** | 216s (3.6m) | 213s (3.5m) | -3s | -1.4% | -| **CI: Tests Storybook** | 103s (1.7m) | 99s (1.6m) | -4s | -3.9% | -| **TOTAL** | **492s (8.2m)** | **486s (8.1m)** | **-6s** | **-1.2%** | - -### Key Findings - -1. **Cache Miss Rate**: The `.cache` directory experienced a **100% cache miss rate** during testing - - Log evidence: `Cache not found for input keys: lint-format-cache-Linux-...` - - This explains why removing it had minimal/no negative impact - -2. **Performance Impact**: Removing `.cache` resulted in: - - **No degradation** - workflows completed in similar or better times - - **Slight improvement** - 1.2% faster on average (6 seconds over ~8 minutes) - - **Consistent behavior** - all workflows showed stable performance - -3. **Cache Effectiveness Analysis**: - - The cache key was based on: `pnpm-lock.yaml` + source file hashes - - With monorepo changes (code moving to `/packages/`), cache keys frequently invalidated - - Build outputs in `.cache` were not being reused effectively - -## Workflow-Specific Analysis - -### CI: Tests Unit (Vitest) -- **Impact**: +1s (+0.6%) - negligible -- **Cache removed**: `.cache` -- **Caches retained**: `coverage`, `.vitest-cache` (tool-specific) -- **Note**: Vitest has its own cache mechanism that's more effective - -### CI: Lint Format (ESLint/Prettier/Knip) -- **Impact**: -3s (-1.4%) - slight improvement -- **Cache removed**: `.cache` -- **Caches retained**: `.eslintcache`, `.prettierCache`, `.knip-cache`, `tsconfig.tsbuildinfo` -- **Note**: Tool-specific caches are more targeted and effective - -### CI: Tests Storybook -- **Impact**: -4s (-3.9%) - noticeable improvement -- **Cache removed**: `.cache` (from both build and chromatic jobs) -- **Caches retained**: `storybook-static`, `tsconfig.tsbuildinfo` -- **Note**: Vite's built-in caching proved sufficient - -## Other Affected Workflows - -The following workflows also had `.cache` removed, though they don't run on every PR: - -- `api-update-electron-api-types.yaml` -- `api-update-manager-api-types.yaml` -- `api-update-registry-api-types.yaml` -- `release-draft-create.yaml` -- `release-pypi-dev.yaml` - -These workflows are triggered on-demand or during releases, and retained their tool-specific caches (`tsconfig.tsbuildinfo`, `dist`). - -## What Caching Remains - -The changes preserve all effective caching mechanisms: - -### 1. Package Manager Caching -- **pnpm cache** via `setup-node` action with `cache: 'pnpm'` -- **Impact**: Saves 2-3 minutes per run -- **Status**: ✅ Retained - -### 2. Tool-Specific Caches -- **ESLint**: `.eslintcache` -- **Prettier**: `.prettierCache` -- **Knip**: `.knip-cache` -- **Vitest**: `.vitest-cache` -- **TypeScript**: `tsconfig.tsbuildinfo` -- **Impact**: Incremental analysis, saves 10-30 seconds per tool -- **Status**: ✅ Retained - -### 3. Build Artifact Caches -- **Storybook**: `storybook-static` -- **Dist**: `dist` directory -- **Coverage**: `coverage` directory -- **Impact**: Prevents full rebuilds -- **Status**: ✅ Retained - -## Why .cache Was Ineffective - -1. **Monorepo Structure Changes**: Code has moved from `/src` to `/packages`, but cache keys didn't account for this -2. **Broad Cache Key**: Hashing all of `src/**/*.{ts,vue,js}` caused frequent invalidation -3. **Vite's Own Caching**: Modern build tools (Vite, Vitest) have efficient built-in caching -4. **Small Delta Time**: Even with perfect cache hits, `.cache` only saved seconds, not minutes - -## Conclusion - -**The `.cache` directory caching should be removed** because: - -✅ **No performance degradation** - workflows run at similar or better speeds -✅ **Simpler maintenance** - fewer cache invalidation issues -✅ **Better cache efficiency** - tool-specific caches are more effective -✅ **Clearer intent** - each cache serves a specific, documented purpose -✅ **Reduced complexity** - fewer cache keys to manage and debug - -The monorepo evolution made generic `.cache` directory caching less effective. Tool-specific caches and pnpm package caching provide better, more predictable performance benefits. - -## Test Data References - -- **Baseline runs (WITH .cache)**: - - Tests Unit: [Run #18624496862](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18624496862) - - Lint Format: [Run #18624496773](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18624496773) - - Tests Storybook: [Run #18624496739](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18624496739) - -- **Comparison runs (WITHOUT .cache)**: - - Tests Unit: [Run #18624644321](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18624644321) - - Lint Format: [Run #18624644316](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18624644316) - - Tests Storybook: [Run #18624644330](https://github.com/Comfy-Org/ComfyUI_frontend/actions/runs/18624644330) - ---- - -*Analysis conducted: 2025-10-19* -*Related PR: #6097* -*Related Issue: #5988*