[GH-ISSUE #303] Failed to run v0.2 using 16Gb card with "CUDA out of memory. Tried to allocate 1.73 GiB" #194

New issue

Closed

opened 2026-10-29 14:35:39 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:35:39 +00:00

Owner

Originally created by @mulinlab on GitHub (Feb 14, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/303

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.73 GiB. GPU 0 has a total capacity of 15.60 GiB of which 954.56 MiB is free. Including non-PyTorch memory, this process has 14.65 GiB memory in use. Of the allocated memory 13.22 GiB is allocated by PyTorch, and 1.16 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

How can I adjust some prams at Linux (eg. Ubuntu)?

Originally created by @mulinlab on GitHub (Feb 14, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/303 torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.73 GiB. GPU 0 has a total capacity of 15.60 GiB of which 954.56 MiB is free. Including non-PyTorch memory, this process has 14.65 GiB memory in use. Of the allocated memory 13.22 GiB is allocated by PyTorch, and 1.16 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) How can I adjust some prams at Linux (eg. Ubuntu)?

bsakaguchi

2026-10-29 14:35:39 +00:00

closed this issue
added the
stale
label

bsakaguchi commented

2026-10-29 14:35:40 +00:00

Author

Owner

@luoning2006 commented on GitHub (Feb 15, 2025):

try export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

@luoning2006 commented on GitHub (Feb 15, 2025): try export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

bsakaguchi commented

2026-10-29 14:35:40 +00:00

Author

Owner

@ErvinXie commented on GitHub (Dec 11, 2025):

This issue has been automatically closed due to inactivity for 60 days.

If you believe this issue is still relevant, please feel free to reopen it with additional information or context.

Thank you for your contribution to KTransformers!

此 issue 因超过 60 天无活动而被自动关闭。

如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。

感谢您对 KTransformers 的贡献！

@ErvinXie commented on GitHub (Dec 11, 2025): This issue has been automatically closed due to inactivity for 60 days. If you believe this issue is still relevant, please feel free to reopen it with additional information or context. Thank you for your contribution to KTransformers! --- 此 issue 因超过 60 天无活动而被自动关闭。如果您认为此问题仍然存在，请随时重新打开并提供更多信息或上下文。感谢您对 KTransformers 的贡献！