mirror of
https://github.com/Comfy-Org/comfy-aimdo.git
synced 2026-11-04 23:37:21 +00:00
[GH-ISSUE #70] Win32 FILE_FLAG_SEQUENTIAL_SCAN has a feature/side effect of discarding data from OS cache after first access #22
Labels
No labels
pull-request
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Comfy-Org/comfy-aimdo#22
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @brendanhoar on GitHub (Jun 29, 2026).
Original GitHub issue: https://github.com/Comfy-Org/comfy-aimdo/issues/70
Presumably this is for @rattus128's attention, correct me if I am wrong.
On windows 32, aimdo's management of safetensors files opens the handle with FILE_FLAG_SEQUENTIAL_SCAN, presumably for performance reasons.
However, a side effect, or feature, depending upon your perspective is that, with this flag, the OS CACHE only caches the data pulled from the file until it is read once, and then it discards the cached data immediately.
In comfyui, I had hoped that --high-ram would have an impact here, but it doesn't seem to change the file handle creation logistics, and that's why I continue to see massive data reads if the models iterated through a single workflow generation use more VRAM than I have (24GB), even though I have 512GB RAM.
Would it be possible to expose controls on whether this flag is used or not, potentially also, whether FILE_FLAG_OVERLAPPED should also be controlled, as it too may have an impact on windows internal OS caching strategies (I haven't looked deep into this yet, but also, I am concerned about out of order IO making it down to physical devices...which may or may not be related)
Presumably these two flags, among others, were used for fastest initial load, perhaps trying to bypass some of the horrible performance issues seen in the past with out of order faulting causing disk thrash, but ... certainly there's a way to ensure that that is dealt with while also not throwing away OS-cached data on systems built specifically to take advantage of them?
Originally, I was investigating FILE_SHARED_WRITE (unsure why this is used) but ran into the above, which appears to be the real root cause of performance issues on my system,
In the end what I would think would be preferable is both allowing the USER with lots of RAM to cache as much safetensors file data as possible at the OS level, while also ensuring that data access patterns to the mapped memory are both large in size and in linear order, so that faults to slower storage don't suddenly incur massive time and hardware thrashing penalties.
Thanks.
B
@VladiCz commented on GitHub (Jun 30, 2026):
I'm on Ubuntu 24.04 and I experience slowdown since ComfyUI 0.22.0 if I remember correctly. My RAM is mostly free, but unloading LTX-2.3 model from VRAM to make space for text-encoder, video decode VAE, audio decode VAE (which is extremely VRAM hungry compared to video VAE), the unloaded model then loads in VRAM very slow, like 100-200 MB/s. This is with 128 GB DDR5 mostly empty and nvme SSD exclusively used for model data files only. I use Lora in every workflow, so maybe Lora have effect on how/from where the model is loaded, not sure.
I added the new comfyui startup option --high-ram, it really eats much more memory, but the speed is identical as without that option, it still loads unloaded model blocks 100-200 MB/s. This is my reading from KJ MemoryVisualisation.
I even disabled barrier (fstab mount option barrier=0) for that nvme model drive to make sure it is not affected by a known Ubuntu SSDs slowdown bug.
@brendanhoar commented on GitHub (Jun 30, 2026):
@VladiCz -
I take all the above back w/r/t my comments on MemoryVisualization tool, what I saw was from the KJ Node VRAM Debugger. Apologies to @VladiCz.
In any case, let's keep focus on aimdo specifically telling Windows OS to discard the safetensors file data from the OS cache as a side effect of the FILE_FLAG_SEQUENTIAL_SCAN flag on CreateFileW.
@VladiCz commented on GitHub (Jun 30, 2026):
That would explain all the issues I have with unloading. But @rattus128 usually requests the output from that node for reporting issues with memory, but that would produce wrong results. If what you write is true, that would be a big issue. I'm going to see the if I will find the default setting to unload models.
@VladiCz commented on GitHub (Jun 30, 2026):
@brendanhoar
So, what you wrote did not make much sense to me. MemoryVisualisation really should not do what you wrote. There is not "default" option to unload models. There is only a single button to unload manually, but that is not what you ment. Could you please send a screenshot of the setting you found as I cannot find any setting for automatic model unloading.
Button to manually unload (not automatic):

@brendanhoar commented on GitHub (Jun 30, 2026):
There's a node added to the workflow under some conditions, let me go it. Ah, it's this one.
Sometimes this was being added to workflows inline without me doing it manually (consciously). I'm not sure why (startup flags? on detect of multimodel workflows? me clicking somewhere on the visualizer?), but I glanced over once and saw all three of these options were enabled, which...obviously made things slower. So I uninstalled the visualizer.
Edit: looks like that node is just part of KJNodes. No idea why it kept getting inserted into workflows. So, nevermind...sorry for sending you down the wrong path? Anyway, back on topic to the CreateFileW flags issue.
@VladiCz commented on GitHub (Jul 1, 2026):
@brendanhoar The MemoryVisualiser is very handy tool so I recommend using all the time as you already discovered, that VRAM Debug from KJ nodes is a very different thing many user will never encounter or only by mistake.
@Aamir3d commented on GitHub (Jul 11, 2026):
cc @comfyanonymous and @Kosinkadink - FYI and not meaning to distract you, but have you seen this? Could this possibly be causing the continuous model load issue?
@anr2me commented on GitHub (Jul 14, 2026):
Reference regarding FILE_FLAG_SEQUENTIAL_SCAN behavior https://devblogs.microsoft.com/oldnewthing/20120120-00/?p=8493
@brendanhoar commented on GitHub (Jul 14, 2026):
Heh, yeah. It kind of pulls aside the curtain on the heuristic shit show going on at the kernel level.
I take it as a cautionary tale to be careful because of the caveats:
So, what I get out of it is: if you specify this flag, you ought to have A/B tested all of the cases (available Ram sizes, a available VRAM sizes, model and sub model sizes, common multi-midel workflows, various storage system/filesystem arrangements, local vs network storage, etc. etc.) before committing to forcing everyone to use the nonstandard heuristics implicit (and somewhat unknowable) buried in the kernel when this flag is active.
ComfyUI allows the end user to specify enabling or disabling of memory mapping of Safetensors and other model files. What I would like to see, at a minimum, would be exposing this createfilew flag as a startup flag under user control, which would allow end users to ad hoc A/B test without a compiler environment and other developer experience needed. This would also mean the aimdo functionality would need to pass it, and the DLL calls would as well.
B
@Bit-Nibble commented on GitHub (Jul 15, 2026):
I think so too. Given the diversity of user environments, I believe providing an option to toggle this (at least for now) is a good idea—even if it is just as a sub-option of
--fast.By the way, here are the results of my testing (on Windows 11 Pro).
When running a generation in a situation where the models cannot be fully held in VRAM and RAM, the model files are naturally re-read repeatedly. However, I observed that if the cache is already full, the system does not discard old cache entries to make room; instead, it repeatedly reads from the physical disk.
What I did:
Launched ComfyUI with the following startup options:
--fast-disk: Prefer disk-backed dynamic loading and offload over unpinned RAM.--disable-pinned-memoryI expected that models would not be held in anything other than VRAM (forcing disk-based offloading).
Performed file access exceeding the available RAM capacity.
The goal was to fill the OS file cache region (I processed numerous large files using 7-Zip's hash calculation feature).
Observed physical disk reads and cache copies using PERFMON while performing two consecutive image generations.


Results:
Every single operation—loading each model, the step-by-step loading of offloaded parts that didn't fit in VRAM, and the model loading for the second generation—was observed to be read from the physical disk.
Replaced


aimdowith a version built without that flag and repeated the test.Result: After the initial model load, the cache was utilized, and no further reads were observed from the physical disk.
Additional Note:
Even with the flag enabled, if there was available space in the cache, the data was indeed being cached.
@VladiCz commented on GitHub (Jul 15, 2026):
@brendanhoar
I have already posted here although I'm on Ubuntu and experiencing exactly the same slowdown.
I have resolved the issue with slow model reads between runs from drive/RAM to VRAM since v0.22.0 by adding --disable-pinned-memory flag. This resulted in stable and fast behavior (up to 10x speedup) I was used to before upgrading ComfyUI from v0.22.0 or maybe v0.21.0, not sure anymore.
I did try multiple flags and combination before. I now removed --fast-disk option I was testing as it did not help in my case even with dedicated nvme PCIE 3.0 drive for models/loras. I kept --high-ram as I have a RAM to spare and it was not utilized properely.
So my relevant startup flags to resolve the issue in Ubuntu are: --high-ram --disable-pinned-memory
Using pinned-memory in all systems as a default option was a bad choice as it is obviously OS or HW dependent as it can results in 10x slowdown with model reads. It can be related to the fact that I only use model + lora combination. I never use bare model without loras be it distill lora, IC-lora, other lora so the model is "constructed" dynamically from multiple lora combination.
Finally, my inference times are longer than model loading/reloding times as for past several months I was only waiting for model to be slowly read back even without changing prompt, seed etc. Now the model is also read back but in matter of seconds, not minutes.
And one more thing for NVME/SSD drive owners. The fact that your NVME/SSD drive was fast when you bought it (or on paper) does not mean it remains fast after few weeks/months/years. Some drives are know to slow down after being written and drop read/write speeds close to SATA HDD, like from 3500 MB/s to 200 MB/s. This includes read, not just writes. This is due to aging of the cells (and it is not even related to trim). The controller can often read the data correctly but with added latency making the drive much slower. "Famous" drives with this issue are Crucial P3, Crucial P3 Plus etc. I also experience this.
I write it because you might be thinking there is no issue with your NVME drive reading the models while it might be one of the factors you might be overlooking. There is plenty more however on the OS level.
@brendanhoar commented on GitHub (Jul 15, 2026):
My system is windows 11 with 512GB of RAM. Models stored on a mix of encrypted nvme and external HDDs.
0.2.2 and earlier, first generations were slow due to storage I/O but subsequent were super fast, and clearly this was due to OS caching the model files. If I pre read the model files once with a tool that did large sequential reads, then as long as I stuck with all of the same models used in a complex workflow, all generations never went back to storage and all ran out of the OS cache limited by GPU only.
Now, with recent comfy, even if I manually read the model files with another tool, after the first gen, comfy goes back to the storage device again. Something is now hinting to windows 11 that the cached files can be discarded at a high priority.
It is most likely that what you are experiencing and what I am experiencing are both happening.
B
@Aamir3d commented on GitHub (Jul 15, 2026):
Also interesting that Linux systems are equally affected. I wish this was sorted as I can't use Comfy in its current state at all.
@Bit-Nibble commented on GitHub (Jul 15, 2026):
@brendanhoar
According to the documentation for CreateFile (https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew):
"Note: When CreateFile opens an existing file, it generally combines the file flags with the file attributes of the existing file, and ignores any file attributes supplied as part of dwFlagsAndAttributes."
This suggests that even for files already residing in the OS cache, the flags passed during CreateFile might be altering the cache management strategy.
@VladiCz commented on GitHub (Jul 16, 2026):
I do not think the issues are equal for Linux and Windows as there are multiple factors, some are relevant for many OS's and some are specific to OS. So there will be no solution fits all. At least I was able to solve the issue with the startup flag, which does not seem to be the case for Windows users.
Also there is the issue of people on Windows who actually run ComfyUI in WSL/WSL2. This brings completely different sum of issues which are related to both Windows and Linux running in container, which has a lot of downsides regarding accessing the resources. I migrated from WSL2 to pure Linux as WSL2 is not a good fit for ComfyUI, maybe from the start, but not in the long run and more heavy use. So for those who are using WSL2 there is really only solution to migrate ComfyUI directly to the host system, preferably Linux.
I really recommend people to dual boot. I started with dual boot and so far not booted back to Windows in 6 months. It remains installed but I do everything on Linux now. Fedora, Ubuntu are a good choice btw.
@kingp0dd commented on GitHub (Jul 16, 2026):
I experience the same, I'm on Linux
https://github.com/Comfy-Org/ComfyUI/issues/14705
@rattus128 commented on GitHub (Jul 22, 2026):
https://github.com/Comfy-Org/comfy-aimdo/pull/76
@brendanhoar commented on GitHub (Jul 30, 2026):
were any wheels/libs built for #76 so that this can be tested in the field?
@anr2me commented on GitHub (Jul 31, 2026):
@brendanhoar commented on GitHub (Jul 31, 2026):
Thanks, unfortunately after doing the embedded\python.exe pip -m install 'path\to\whl'
I get the following on startup:
@anr2me commented on GitHub (Aug 1, 2026):
Looks like it's because artifacts are usually using text letters in the version (ie.
dev) instead of just numbers. WhileVersion.Parsein SwarmUI'sComfyUISelfStartBackend.cscannot process text letters likedev.So the version handling probably need to be fixed first in order to support this kind of versioning.
@brendanhoar commented on GitHub (Aug 1, 2026):
I'll retest a bit later, mcmonkey said it's only during autoupdate, which can be disabled for this purpose.
@anr2me commented on GitHub (Aug 1, 2026):
Besides
devthey might want to handleposttoo while at it 😅 because the text "post" often used in many nightly wheel version.@brendanhoar commented on GitHub (Aug 30, 2026):
@rattus128 -
This appears to be resolved from my perspective, I forgot to post here when it occurred.
I saw that you'd merged the recommended change into comfy-aimdo which then flowed to comfyui. At which point the issue fully resolved for me.
I can't say for SURE whether this recommendation was THE fix, but it was one of three commits in aimdo that went out that day, and the problem went away that day, so...probably?
Anyway, thanks for addressing it!
B