mirror of
https://github.com/Comfy-Org/ComfyUI_frontend.git
synced 2026-03-14 09:27:41 +00:00
test: add property-based FSM tests for workflow persistence (#9370)
## Summary - Fixes #9319 - Add [fast-check](https://github.com/dubzzz/fast-check) property-based testing with FSM (Finite State Machine) traversal to automatically explore state combinations in the workflow persistence system - Fix a real bug in `saveDraft()` discovered by the FSM test: orphan cleanup in `loadIndex()` could delete a just-written payload when the in-memory cache was empty ## Why this is needed #9317 exposed a class of bug where two independently correct changes interact to cause workflow loss. Conventional unit tests verify specific, hand-picked scenarios and cannot catch these cross-PR interaction bugs. ### AS IS (before) | Aspect | Status | |---|---| | Testing approach | Example-based: developer picks specific inputs and expected outputs | | State coverage | Only explicitly written scenarios are tested | | Cross-interaction bugs | Not detectable — each test runs one isolated path | | Bug in `saveDraft` | Undetected — `loadIndex()` orphan cleanup could delete a just-written payload after `reset()` | ### TO BE (after) | Aspect | Status | |---|---| | Testing approach | Property-based: fast-check generates **200 random command sequences** per run | | State coverage | Random exploration of `SaveDraft → GetDraft → RemoveDraft → MoveDraft → GetMostRecentPath → Reset` combinations | | Cross-interaction bugs | Detected automatically — fast-check shrinks failing sequences to minimal reproductions | | Bug in `saveDraft` | Found and fixed — `loadIndex()` now runs **before** `writePayload()` to prevent orphan cleanup race | ## What fast-check does fast-check is a property-based testing library. Instead of testing "does this specific input produce this specific output?", it tests "does this **property** hold for **all possible inputs**?" For FSM testing specifically, fast-check: 1. Takes a set of **commands** (SaveDraft, GetDraft, RemoveDraft, MoveDraft, GetMostRecentPath, Reset) 2. Generates **random sequences** of these commands 3. Runs each sequence against both a **model** (simplified oracle) and the **real system** (store + localStorage) 4. Verifies **invariants** after every mutating command (index/payload consistency, no orphans, LRU correctness, model agreement) 5. When a failure is found, **shrinks** the sequence to the minimal reproduction Example: the bug this PR fixes was shrunk to just 4 commands: ``` SaveDraft(d.json) → RemoveDraft(d.json) → Reset() → SaveDraft(a.json) ✗ ``` ## Changes | File | Change | |---|---| | `package.json` / `pnpm-workspace.yaml` | Add `fast-check` devDependency | | `draftCacheV2.property.test.ts` | 7 property tests for pure index functions | | `workflowDraftStoreV2.fsm.test.ts` | FSM test: 6 commands, invariant checking, 200 runs | | `workflowDraftStoreV2.ts` | Fix: move `loadIndex()` before `writePayload()` in `saveDraft()` | ## Test plan - [x] `pnpm test:unit` — all 117 persistence tests pass (including 7 property + 1 FSM) - [x] `pnpm typecheck` — clean - [x] `pnpm lint` — clean - [x] Pre-commit hooks pass (format, lint, typecheck) - [x] Pre-push hook passes (knip) ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-9370-test-add-property-based-FSM-tests-for-workflow-persistence-3196d73d3650813daa98cdd8bef7e975) by [Unito](https://www.unito.io) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: GitHub Action <action@github.com>
This commit is contained in:
@@ -154,6 +154,7 @@
|
||||
"eslint-plugin-storybook": "catalog:",
|
||||
"eslint-plugin-unused-imports": "catalog:",
|
||||
"eslint-plugin-vue": "catalog:",
|
||||
"fast-check": "catalog:",
|
||||
"fs-extra": "^11.2.0",
|
||||
"globals": "catalog:",
|
||||
"happy-dom": "catalog:",
|
||||
|
||||
Reference in New Issue
Block a user