Pack-mode breaks are delivered via manifest pin swaps (CI clones packs at
pins, so in-repo pack-file edits cannot reach it); corpus-derived red
messages promise the tier and failure class, not byte-identical offender
text; remaining citations and mechanism descriptions tightened to what the
code and captured runs actually show.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every quoted red message re-verified against its original run log (10/10
match). Seven citation corrections from the audit: the iTools missing-
button tickets are Nodes 2.0 regressions and move to the v2 mount row
(the v1 row is now honestly class-only); the SAM3 hidden-values tickets
are removed (extras-exposed class, which the mount tier tolerates by
design, so citing them overclaimed coverage); the persistence row now
cites the verified defaultInput migration regression (widgets reverting
to socket-only on reload) that open PR #12279 fixes, instead of a live
widget-interaction ticket; the links-type and serialization rows drop
borrowed tickets and state their class plainly; the drag row's tickets
are labeled nearest-symptom family; the two expansion bullets now cite
the committed pure-spec catches instead of an unproven live-sweep catch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every test gets a fresh page but all tests share one backend, locally and
on CI alike (the CI job is deliberately unsharded). A test ending while
its prompt still executed left that work running, and the next test's
fresh page connected mid-execution and inherited its async errors or its
busy queue. Drain the backend to idle in an afterEach in all four
backend-running specs, while the finishing test's own page is still open
so late events land there: no test can affect the next.
The drain helper moves to the shared fixture util (drainBackendToIdle,
byte-identical body); the auto-run tier's queue guard and runBatch
post-timeout drain rewire to it with their explicit budgets. The hooks
use a 10s budget: a no-op when already idle, and a backend still busy
past it is wedged, which the auto-run tier's 150s guard surfaces with
the restart diagnostic.
DETECTION_PROOF.md's caveat is rewritten to match, and a false claim
that CI shards one backend per pack is corrected in every location
(code comments and doc): the CI job runs the whole suite against one
fresh backend on an unloaded runner, which is why executions stay
inside their budgets there.
Empirical: a full-suite run with the hook eliminated the cross-test
bleed class entirely (zero mount/save-reload/core-smoke console or
overlay failures, previously 3-5 per run).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The suite runs all 7 packs' execution tiers against one shared backend
locally (CI shards one backend per pack). Serial execution created three
distinct cross-test contaminations that failed a different set of packs
each run; each is now fixed at its mechanism:
- Foreign execution noise: mount/persistence/wiring/T0/core-smoke tiers
queue no prompts, yet caught a prior tier's async execution error
(PromptExecutionError, a 400 on /api/prompt). isForeignExecutionNoise
filters execution-domain console lines from the non-executing tiers
only; the executing tiers still assert them. Same "not this test's
evidence" principle as event attribution (ARCHITECTURE section 9).
- Queue contention: the auto-run queue-busy guard hard-failed when a prior
pack's slow CPU execution was still draining. drainUntilIdle waits it
out (interrupt + clear + poll, throw-on-error so a failed read counts as
busy); only a genuinely wedged backend fails. runBatch's post-timeout
drain grows from 5s to 90s for the same reason.
- Slow-under-load misread as a regression: the single-node disambiguation
re-run gets 60s instead of the batch's 20s. A real hang still exceeds it.
Also excludes the CLIPSeg model loaders (essentials, WAS) - model-download
nodes, same non-interruptible class as the listed BLIP/SAM/MiDaS loaders.
Reviewed by four-hat CORE (ship it); the new predicate is unit-pinned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The correlation matrix + throwaway-PR plan that proves the suite catches
every failure mode ARCHITECTURE.md claims: one deliberate break per
surface, each citing the real historical regression it recreates (Linear
Custom Node Bugs issues + FE PR #12279) and the exact CI red it produces.
Every "exact red" is captured from a real falsification run, not a
prediction. Renames the earlier "kill-test" work to the falsification
pass. States the honest local-full-run idempotency caveat (CI shards
per-pack; a single-backend serial run is not the oracle).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Wiring drop resolution: the curated drag test only targeted first-slot
inputs, so a slot hit-test regression that falls back to the first
compatible input went undetected. Add a second-slot anchor
(EmptyImage.IMAGE -> ImageBatch.image2) that only links if the drop
resolves the exact slot; proven by breaking getNodeInputOnPos.
- Curated-run failure naming: a backend validation rejection answers
/prompt with node_errors but app.queuePrompt swallows it, so a
VALIDATION_FAIL reported {}. Capture and flatten the node_errors
(summarizePromptError, typed off apiSchema PromptResponse) into the
result's clientError and surface it in the T1 message, so a red names
the node and input. Exported with a pure unit test since the happy path
never runs it.
- Console-error window: document (README + ARCHITECTURE section 10) that
the ledger collects per-tier, so boot-time pack console noise before the
first tier action is out of scope by design, backstopped by the startup
zero-visible-errors check.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Round-3 review follow-ups: the definition-pipeline diagram no longer
implies a centralized normalizer (live census -> wiring slot normalizer /
execution classifier / mount declared-shape parser, matching section 4),
and ADDING_CUSTOM_NODES scopes the zero-visible-errors claim to the tiers
that assert it, same wording as the README.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Extract the pack console-error allowlist into a shared fixture
(consoleErrorLedger.ts); the curated T1 run now collects console and
page errors across load+run and asserts them through the ledger. The
filter is pinned by a discriminating pure spec (pattern match,
cross-pack ownership, unknown-pack fail-open).
- T1 asserts every expectedNodes type is actually present in the curated
workflow before running it, killing the vacuous-green path where a
drifted fixture shrank the executed-set check to an empty id list.
- typePairing records unrecognizable slot specs (unknownSlots on the
node, unknownShapes on the plan) instead of silently dropping them;
connectivity logs the list; pure tests pin the input/output drop paths
and the socketless boundary.
- Add test:custom-nodes:ci, the gate-equivalent run against the
backend-served built frontend; README re-scopes test:custom-nodes as
the dev-server loop that is NOT the gate, and scopes the
zero-visible-errors invariant to the tiers that hold it.
- ARCHITECTURE truth pass: event attribution leads with the positive
prompt-id capture; section 10 grades ledger guards in three strengths;
the decentralized parser story (declaredShape, classifyInput,
normalizer) is stated consistently in section 4, gotcha G5, and the
legend; ADDING_CUSTOM_NODES points the console ledger at its new home.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- positive prompt-id attribution: capture the /prompt response id as the
primary event filter (seen-set + graph membership stay as depth); a new
attribution self-check injects a foreign-prompt terminal error mid-run
and proves it cannot fail the run
- console collection now includes pageerror (uncaught exceptions and
rejections), with a collector self-check as positive control; surfaces a
real Custom-Scripts betterCombos typeof-null bug, ledgered with mechanism
- connectivity allowlists are two-way stale-guarded: every entry must be
observed failing in its recorded way, all stale keys reported per run
- manifest pins are required full 40-hex SHAs (CUSTOM_NODES_ALLOW_UNPINNED=1
admits only empty pins, reserved for the planned pack-HEAD canary); pack
must be a plain path segment; contract pinned by pure specs
- CI installs each pack under custom_nodes/<pack> with charset and pin
gates before cloning (attribution keys on the install dirname)
- allNodes renderer loops honor rendererPassesFor (vueNodesCompatible)
- curated T1 asserts every display sink emitted a ui payload; console
sinks documented as excluded (no ui payload by design)
- the always()-wrapper suggestion was rejected on sibling evidence:
ci-tests-unit.yaml gates its required check with a changes job and
job-level if, and no repo workflow uses a wrapper
Reviewed via ninja pipeline: 4-hat CORE panel (2 passes), senior QA gate
(2 rounds, discrimination proven by falsification), gated review (Primary,
Double Checker, Ultimate Skeptic - 15-entry evidence ledger, all PROVEN).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dagre's crossing minimizer ignores edge declaration order, so the fix is
node declaration order. Also states explicitly that the map is ordered by
zoom, not page order.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- context diagram flows one way: driver -> frontend -> verdict synthesis -> team
- execution flow: classification fans out to its three verdicts; runnable paths
converge on batching, blocked routes straight to reconciliation
- persistence check: sequence diagram replaced with a linear pipeline (one
actor issuing commands is a procedure, not a message exchange)
- building blocks: tiers fan 2x2 inside the horizontal pipeline
- tripwire step + small recovers? diamond instead of one giant diamond
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The building-blocks pipeline rendered as a tall narrow strip for two
mermaid reasons: labels auto-wrap at the ~200px default regardless of
line length, and a subgraph's declared direction is ignored once it has
external edges, so the tier row silently stacked vertically. The view
is now a left-to-right pipeline with the tier group in the middle, and
the wrap-width directive makes boxes wide instead of tall here and in
the definition-pipeline and execution-flow views.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The frontend box was a dead end: the suite drove it but nothing flowed
back, so the verdicts arrow to the team looked sourceless. Added the
return edge (observations back: what mounted, what persisted, what
executed, every error) and reworded the team edge so verdicts are
visibly the synthesis of those observations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Headings were double-numbered ("2. D1 - system context") with an
internal diagram-numbering scheme that means nothing to a reader.
Sections are already numbered: headings now just name the view, and
every cross-reference points at a section. Also replaced "queue cost
is amortized" with plain English: one submission carries many nodes
instead of paying the round-trip per node.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reader feedback on the rendered views: manual line breaks inside boxes
force Mermaid to render narrow, tall boxes with heavy wrapping, so the
diagrams cost too much scrolling. Mermaid sizes a box to its longest
line, so the fix is one or two long lines per box with elaboration in
the prose below the diagram. Applied to D1 (context), D2 (pipeline),
D3 (definition pipeline), and D4 (execution flow); D7 and D8 stay as
approved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reader feedback on the rendered views: D2's service-to-tier arrows
crossed the whole diagram with ambiguous fan-ins (three unattributed
"pass/fail + exceptions" curves), and D3's corpus box mixed the
two-dialects annotation into a flow node right where three arrows fan
out, reading as if the dialects explained the fan-out.
Fixes, using the rules that make the CI view work: one direction of
flow per diagram, no many-to-many edges (the service-to-tier matrix is
now a table, which is what a matrix is), and annotations live in prose
rather than inside flow boxes. D2 is now a straight
manifest -> orchestrator -> tiers -> evidence -> verdict pipeline with
a three-row shared-services table; D3 moves the dialect fact into the
normalize step, labels the fan-out "derives", and adds one sentence
mapping each derived plan to its consuming tier.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Architecture documentation for the custom-node regression suite,
written as design views rather than an implementation dump:
- eight responsibility-level views: system context, building blocks,
the node-definition pipeline, the execution flow, the persistence
check, event attribution, the evidence model, and the CI deployment
view; every diagram box names a responsibility or concept, arrows
carry meaning, and decision points read in plain English
- a one-minute What/Why/How opening with the three explicit non-goals
(output semantics, frontend-virtual nodes, hour-scale soak) and a
clearly labeled scale snapshot so instance numbers never read as
properties of the design
- a 12-row design-decisions table with honest trade-offs (why a real
browser at all, why the backend serves the built frontend, one
worker, disabled execution cache, pinned pack versions, one-row
extensibility, per-tier renderer policy, mechanism-carrying
exceptions, the two-way baseline, batch+bisect, and the scope line),
plus the curated-workflow fixture named as the deliberate extension
seam
- a 14-item gotchas reference, each entry in symptom / root cause /
defense / which-team-concern-it-answers form, with named nodes kept
only as worked examples of their class
- one implementation map section where architecture names meet code
symbols, covering every building block including the orchestrator
and the evidence ledgers
- the workflow's rotted sharding comment fixed (suite duration and
the real shard trigger)
Grounded on the C4 model's published guidance, reviewed by an
independent architect pass (two view-coherence gaps found and fixed)
after three earlier Opus review passes on content accuracy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The suite had run/onboarding docs but nothing describing the SYSTEM:
what the pieces are, how node definitions flow through the planners and
classifiers, how the execution harness attributes outcomes, and why each
non-obvious rule exists. This adds the missing third doc with four
Mermaid box-line diagrams (system overview, def data flow, run pipeline,
CI pipeline), the tier-by-renderer coverage matrix, the full ledger
table with the two-way baseline semantics, and the hard-won invariants
each tied to the incident that forced it (widgetValueStore id bleed,
event cross-attribution, pack JS queue-hook crashes, Vue effect timing,
queue-jam tripwire). Scope contract is stated up front: compatibility
and regression gate, not a behavior certifier.
Every path, symbol, and number cross-checked against the tree before
commit. README and ADDING_CUSTOM_NODES now cross-reference it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Systematic audit for siblings of the two combo bug classes (ungrounded
contract, shape blindness), driven by a live shape census of the exact
getNodeDefs object the suite consumes:
- classifyInput now handles the V2 schema form (string 'COMBO' with
options in the opts object; 495 such inputs exist in the transformed
defs): options present = widget, empty or remote/lazy = NEEDS_MODELS.
Real effect measured: 8 KJNodes nodes were silently misclassified
NEEDS_WIRES and never executed - 5 now run clean, 3 correctly land in
NEEDS_MODELS (remote combos).
- forceInput now beats every input form, list-form combos included (a
census-found form the old branch order classified as widget; today's
4 instances are optional or non-manifest, so this is protection, not
a behavior change).
- pure-spec fixtures for both parsers now include every census form,
copied from real census examples (V2 options, V2 empty, V2 remote,
forceInput-on-combo, cross-form vocabulary pairing) so fixtures can
no longer self-confirm the parser's assumptions.
- ADDING_CUSTOM_NODES.md gains the evidence rules: independent-oracle
grounding for semantic claims, shape-census-driven parsing with
exclude-with-record on unknown shapes, and verify-against-the-source-
the-code-consumes.
defaultInput checked against frontend source: deprecated and ignored
(nodeDefStore warning only) - deliberately not handled.
Local verification: full customNodes suite 72/72 under CI parity; lint,
format:check, knip, and both typechecks clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A wired combo input bypasses its own widget, so menu order and the
options[0] default are not part of the wire contract - membership is
(backend validation checks value-in-options). Vocabulary fingerprints
are now order-insensitive (sorted, element-wise canonicalized). In the
current corpus this changes zero pairs (measured: no same-set,
different-order combos exist across the 7 packs); the rule is now
correct for packs where they do.
Auditing that change surfaced a real hole: the frontend's transformed
defs present some combos as the literal string COMBO with options in
the opts object. The old fingerprint hashed all of those identically,
silently cross-pairing dropdowns with no vocabulary evidence - exactly
the checkpoint-into-scheduler class the combo rule exists to exclude.
Normalization now pulls V2-form options, and a combo with no known
option list is excluded from pairing instead of blind-matched. Plan
moves 5,058 -> 5,030 pairs; the 28 removed were vocabulary-blind.
Also from CI: MiDaS Mask Image excluded (torch.hub download inside
execute hung the Linux runner; runs clean only where the hub cache is
warm) and ImageTransformKJ ledgered (pack JS initializes its
fill-options JSON widget on configure).
Local verification: full customNodes suite 68/68 twice, lint, format,
knip, and both typechecks clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Coverage-gap audit after the mount-fidelity miss (assertions silently
narrower than their claim). Fixes:
- save/reload now runs under BOTH renderers with staged evaluates and
frame yields, so Vue component mount/configure effects actually flush
before each serialize - the one renderer-dependent value path a
LiteGraph-only pass could not see. Console errors are now collected
during the tier too (configure-time pack JS noise was uncovered).
- queuePrompt is wrapped in-page: pack JS that THROWS mid-graphToPrompt
(VHS applyToGraph crashed CI's whole VHS tier) now classifies as
VALIDATION_FAIL carrying the exception text, so the offender
self-identifies instead of aborting the tier. VHS_SelectLatest
excluded with that mechanism: its applyToGraph assumes downstream
inputs have widgets and hard-crashes when its output feeds a pure
socket while the input dir has matching files (upstream-report
candidate).
- pack-owned-value nodes (ROUNDTRIP_VALUE_ALLOWLIST) no longer receive
set-and-stick probe writes - writing `_cn` markers into editor JSON
widgets just made pack JS choke on our own probes.
- deliberate scopes are now stated where the assertion lives: auto-run
runs single-renderer because execution is a backend contract and
values flow through the same store in both renderers; it deliberately
skips the zero-visible-errors check because it provokes expected
failures; the connectivity breadth sweep is renderer-independent with
the curated drag test covering both renderers; combo vocabulary
matching is deliberately order-sensitive (option order defines the
default).
Local verification: full customNodes suite 67/67 under CI parity, plus
lint, format:check, knip, and both typechecks clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mount fidelity now has a renderer-independent bar: under BOTH the
LiteGraph and Vue passes, every created instance must materialize
everything its def declares - each non-socketless input exists as a
widget or a socket (autogrow templates count via their dot-qualified
expansion slots, e.g. variables.a/variables.b), and every declared
output exists. The Vue pass keeps its extra layer: the DOM must render
at least the instance's widget and slot counts. Verified against all
823 nodes under both renderers; the only def-shape special case found
was the core autogrow container semantics.
Also reconciles the first Linux CI run of the chain-builder tier:
- environment flips move to AUTO_RUN_EXCLUDE with mechanisms and leave
the baseline: Image Analyze, Text Parse A1111 Embeddings (fail macOS,
clean Linux), Image Crop Face (clean macOS, AttributeError Linux),
ImageReceiver (av decode error macOS, clean Linux)
- run-to-run flip-floppers excluded: ImpactRemoteInt,
ImpactSchedulerAdapter, ImpactQueueTriggerCountdown (queue-hook JS
transient refusals), LoadText|pysssss (state-dependent file combo)
Local verification: allNodes 21/21 twice consecutively, plus lint,
format:check, knip, and both typechecks clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Executes ~340 more nodes and hardens every tier's assertions:
- CHAINABLE verdict: required sockets with a model-free producer
(EmptyImage, EmptyLatentImage, SolidMask, Primitive*, EmptyAudio) are
synthesized and wired automatically; NEEDS_WIRES now means only truly
unproducible types (MODEL, SEGS, CONDITIONING...)
- auto-run asserts data flow: every PreviewAny sink must emit a ui
payload (NO_OUTPUT class); OUTPUT_NODE targets stay event-covered
- Vue mount asserts DOM widget/slot counts (missing fails, extras and
in-row control_after_generate tolerated)
- save/reload is now two passes: pristine (reload must never shrink a
node or change a value - the "widgets disappear" bug class) and
set-and-stick (every plain widget holds a programmatic non-default
write and it survives reload where topology is stable)
- connectivity pairs COMBO slots on exact option-vocabulary match
(+~120 pairs); mismatched vocabularies stay excluded by design
- harness invariants: node ids never reused within a page (the
widgetValueStore keys state by node id and survives graph.clear(), so
a reused id inherits stale widget values - core bug, reported
separately), and run events are filtered by prompt id + graph node id
membership so late websocket events or flap-retry double-queues can
never pin one node's failure on the next
- new mechanism ledgers: WIDGET_SET_ALLOWLIST, ROUNDTRIP_VALUE_ALLOWLIST,
MOUNT_WIDGET_ALLOWLIST, all stale-guarded; AUTO_RUN_EXCLUDE gains the
observed offenders (rembg pip-install-at-execute, empty-find infinite
loop, from_pretrained downloads, minutes-long per-pixel loops)
- manifest baselines reconciled against three observation runs; stale
entries removed, real failures (missing optional deps, degenerate
synthesized inputs, CUDA-only recorders) baselined
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ImageGrabPIL is a KJNodes node and was ledgered under WAS, so its
exclusion never applied - moved to the right pack, and the auto-run
test now asserts every exclusion key is actually registered by its
pack so a wrong-pack entry fails loudly instead of silently doing
nothing. WAS Create Video from Path joins the exclusions with CI
evidence (ffmpeg discovery differs per host).
The rebuilt environment surfaced the rest of the download class: WAS
BLIP/SAM/MiDaS model loaders all hang the queue in non-interruptible
weight downloads (and would mass-download on a networked runner), the
random.org node needs internet by definition, KJ LoadAndResizeImage and
WAS Create Grid Image follow input-dir contents, and Impact media
widgets preview values via root-relative URLs (console allowlist
widened). Every entry carries its mechanism; all keep mount,
save/reload, and connectivity coverage. 65/65 both environments.
typecheck:browser (which the lint job runs, unlike root typecheck)
caught an unused classifyInput param, a branded-NodeId lookup, and
api.interrupt's required argument. The first serialized CI run then
exposed environment-variable nodes: clean on one host, failing on the
other (screen capture with no X display, PIL screen grabs headless,
torch-stack RuntimeErrors that are macOS-only, state-dependent WAS
history). Those move from the cannotRunAlone baseline to
AUTO_RUN_EXCLUDE with per-node mechanisms so both environments stay
deterministic, and Impact's hardcoded example.png preview 404 joins
the scoped console allowlist.
vitest/valid-expect flags assertions stored before advancing fake
timers and awaited after - awaiting at creation would deadlock the
timer advance. The file is byte-identical to main; the warnings appear
because the type-aware lint toolchain moved. Suppressed per line with
the reason; also drops a scratch await added while chasing this.
The auto-run tier needs exclusive backend-queue access, so the CI job
now runs with workers=1 - parallel workers were interrupting each
other's executions and cross-attributing errors. Await the async
toHaveLength assertion the type-aware lint flagged, drop the
calibration measurement harness (one-shot scaffolding; the measured
constant keeps a provenance note), rename NO_SINK to
NO_OBSERVABLE_OUTPUT, and cut comments down to load-bearing WHYs.
An unpinned pack means any upstream push can red the gating check for
every PR in the repo. Each row now pins the exact SHA the suite was
verified against locally (all tiers green, both environments); bumps
are deliberate, re-verified changes. Also records why the job is not
sharded yet: per-shard setup (~4.5 min of pack installs and backend
boot) dominates the ~5.5 min suite, so a prebuilt image comes first.
The cannotRunAlone baseline (per pack, in the manifest) records nodes
that cannot execute standalone on a bare backend, asserted both ways so
entries cannot rot: an unlisted failure is a regression, a listed node
that runs clean must be removed. queuePrompt rejection is retried once
before classifying VALIDATION_FAIL - pack JS hooking the queue path can
refuse transiently, and the backend log proved several apparent rejects
never reached the server. Nodes whose execution depends on their own
pack JS preprocessing widget values (rgthree Power widgets, KJ editors)
are excluded unconditionally with the mechanism recorded, since whether
a page applies pack JS varies by serving setup; ML-session initializers
and unstable executed-set reporters join them. ADDING_PACKS and the
README document the every-node tiers and all five exception ledgers.
Verified 65/65 in both documented environments: dev server and
dist-serving CI parity, twice consecutively on the latter.
All-nodes tiers discover each pack's full node list from the live
backend: chunked mount checks in both renderers (batch size 24, chosen
by the committed calibration tool), chunked save/reload round-trips,
a connectivity corpus widened from the curated sentinels to every
registered node, and an auto-run tier that classifies every node
(AUTO_RUNNABLE / NEEDS_WIRES / NEEDS_MODELS / NO_SINK) and executes the
runnable ones in batches with per-node bisection on failure.
Hard-won harness rules baked in: queuePrompt rejects classify instantly
as VALIDATION_FAIL instead of burning the timeout; a timed-out batch
interrupts and verifies the queue drained so one hung node cannot jam
every later run; a pre-flight queue check fails fast with the real
cause; and three reviewable exception ledgers carry reasons inline
(AUTO_RUN_EXCLUDE for runtime-downloaders like RemBGSession+, a scoped
console-noise allowlist for KJNodes' undefined-filename previews,
connectivity CONNECT_REJECTED/ROUNDTRIP_LOST entries for pack JS that
vetoes or drops links).
## Summary
Follow-up draft PR for the CodeRabbit issues created from the #12999
review. This keeps the original stabilization PR merged as-is and moves
the non-functional TemplateHelper cleanup into its own small branch.
## Changes
- Extracted TemplateHelper route patterns into named module-scope
constants.
- Normalized the TemplateHelper route patterns to anchored regexes with
optional query-string handling.
- Extracted `mockCustomTemplates()` from `mockIndex()` and made `mock()`
register custom templates, core index, and thumbnails together.
- Added a private `registerRoute()` helper so every mocked route is
registered for teardown consistently.
- Simplified the fixed empty custom-template response to `body: '{}'`.
- Updated the cloud template filtering spec to use `templateApi.mock()`
instead of manually combining thumbnail and index mocks.
## Issues
- Closes#13014
- Closes#13016
- Closes#13017
- Closes#13018
- Related #13015: this PR normalizes the TemplateHelper route patterns
only. The broader fixture-wide route pattern convention cleanup remains
intentionally separate.
## Validation
- `pnpm exec oxfmt --check
browser_tests/fixtures/helpers/TemplateHelper.ts
browser_tests/tests/templateFilteringCount.spec.ts`
- `pnpm exec eslint browser_tests/fixtures/helpers/TemplateHelper.ts
browser_tests/tests/templateFilteringCount.spec.ts`
- `pnpm typecheck:browser`
- Pre-commit hook also ran `oxfmt`, `oxlint`, `eslint`, `pnpm
typecheck`, and `pnpm typecheck:browser` successfully.
Note: I attempted the targeted cloud Playwright spec locally with
`PLAYWRIGHT_LOCAL=1 PLAYWRIGHT_TEST_URL=http://localhost:5173 pnpm exec
playwright test browser_tests/tests/templateFilteringCount.spec.ts
--project=cloud`, but the local 5173 app was not running with the cloud
distribution configuration, so the distribution-filter assertions failed
in the expected local/cloud mismatch way. This should be verified by
CI's cloud project.
grep -c on the single-line /extensions JSON could only say 0 or 1; count
entries properly with the same python one-liner style the doc already
uses. The Step 7 triage claimed our CI failure was upstream drift; it was
the dev-server blindspot - reorder the advice to reproduce under 6b
before diagnosing.
Detect frontend-JS packs at install time, split local verification into
the fast dev-server loop and the CI-parity dist run (required when the
pack ships frontend JS), spell out that workflow media paths resolve
against the backend's working directory, and add upstream-drift triage
for unpinned packs. Checklist updated to match.
Two CI failures with the 7-pack backend, both from pack frontend JS that
never loads under the Vite dev server (its /extensions list is core-only):
- rgthree's Seed rebuilds its declared seed input as a widget-only
control, so the planned BatchCount+.INT -> Seed.seed pair has no socket
on the instance. The sweep now classifies that as
WIDGET_ONLY_ON_INSTANCE, logged and excluded like wildcards; a name
missing from both slots and widgets still fails hard, and the drag test
picks the first in-pack pair that materializes on real instances.
- rgthree's progress bar shifts the canvas element 16px down, and
NodeSlotReference.getPosition returned canvas-relative coordinates, so
every slot drag grabbed the node title instead of the slot dot. Slot
positions now include the canvas element's page offset (a no-op when
the canvas sits at 0,0).
Documents the dev-server blindspot and the CI-parity loop (build dist,
--front-end-root) in the suite README and ADDING_PACKS. Verified 36/36
green against both the dev server and a dist-serving 7-pack backend.
A run row with an empty workflow would skip locally and rely on CI's
skip gate to notice the lost coverage; enforce the documented contract
at load time instead.
Five new manifest rows, each covering load, connectivity, and run tiers
with hand-authored model-free workflows verified against a live backend.
New optional vueNodesCompatible manifest field: a pack proven unable to
mount under Vue Nodes 2.0 runs its LiteGraph assertions only - never a
test.skip, so the zero-skip CI gate stays honest. All five packs mount
under Vue Nodes 2.0 empirically, so no row sets the flag; the decision
helper is unit-tested instead. ADDING_PACKS.md is the authoritative
step-by-step onboarding process, validated against live /object_info.
Manifest rows now also fail fast on an empty repo field.
Derive CustomNodeTier from the VALID_TIERS array (as const) so adding a tier
is one edit and the type/runtime lists can't drift. The forbid-skips
diagnostic now recurses the report and prints only specs that actually
skipped - the old dump printed every title and a single-level filter would
miss specs nested under describe() blocks (which the regression spec uses).
Security: the pack-install job now runs only for same-repo PRs and pushes, so
a fork PR can't point the manifest's repo URLs at attacker-controlled code
that the job would clone and pip-install. Fork PRs keep the env-agnostic
coverage via the main e2e shards.
Stability: pack requirements install under a pip constraint pinning the CPU
torch stack, so no pack can swap torch for a GPU/incompatible build on the
--cpu runner.
Correctness: manifest validation rejects unknown tier values (a 'connectivty'
typo would otherwise silently drop that tier's coverage). Connectivity's
'pack installed' predicate is extracted to one isEntryInstalled helper used by
both the breadth and drag tests.
T-conn was planning-doc shorthand for the connectivity tier; test titles and
logs now say connectivity outright so CI output reads without tribal
knowledge.
CI caught what a pack-rich local backend masked: isValidConnection compares
only the string COMBO while every combo slot carries its own option set, so
the planner would wire a checkpoint dropdown into a scheduler dropdown and
call it proof, and combo outputs declare a non-string output_name whose
instance slot name never matches (DevToolsNodeWithOutputCombo failed 5
pairs on CI as SLOT_CONTRACT_MISMATCH). Combo slots are now recorded and
counted like wildcards instead of paired, the normalizer coerces slot names
to strings, and a pure spec locks both behaviors. Targeted fixtures remain
the way to cover combo semantics.
The Comfy.userId=default settings override broke every test on multi-user
backends (the repo's stated browser-test prerequisite): devtools
set_settings wrote to a user no session reads, so Comfy.TutorialCompleted
never landed, the templates dialog never opened, and the beforeEach wait
timed out - CI sessions even inherited leftover settings (a zh locale) from
earlier tests on the same worker user. Dropping the override lets the
fixture target the real per-worker user everywhere; the harness backend now
runs --multi-user like CI. Connectivity's per-pack guards and drag
derivation apply only to installed packs, so a backend without the manifest
packs reports the absence instead of hard-failing while the core sweep,
native drag, and self-checks still run.
knip flags exported types with no external consumers; CustomNodeTier,
ObjectInfoNode, NormalizedSlot, and SlotRef are referenced only within
their own modules.
