[GH-ISSUE #105] Some supplementary on `ld.nc` in PTX model and texture memory for pragmatic guess in README #94

New issue

Closed

opened 2026-10-26 09:54:11 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-26 09:54:11 +00:00

Owner

Originally created by @zsnoob on GitHub (Apr 1, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/105

https://github.com/deepseek-ai/DeepEP/issues/21#issuecomment-2764894125
This comment refered that: 'nc encourages NVCC to do more optimizations.' I'm so curious about the hardware side of this architecture-specific characteristic beyond compiler optimization, so I did some summary from docs. Hope others who are also interested in this detail will find the information helpful and get some inspiration for own program design.

Though we disable the texture cache's allocation, the behavior how it process access requests may still occur.

it's designed to be a "bandwidth aggregator" which aggregates simultaneous loads request from many threads and tries to stream the results back to the processing units as efficiently as possible, rather than L1 cache that designed for latency.

As the texture cache offers very different coalescing rules over the data cache, using this memory path may improve performance - especially for highly non-uniform memory access patterns.

To accelerate memory access, GPUs typically rely on coalescing. However, this optimization comes with certain constraints. The texture cache mechanism can be used to get around coalescing constraints. We have better performance when data access is out of coalesced constraints and have sufficient parallelism.

Reference:

https://docs.nvidia.com/cuda/parallel-thread-execution
https://docs.nvidia.com/cuda/cuda-c-best-practices-guide
https://docs.nvidia.com/gameworks/content/developertools/desktop/analysis/report/cudaexperiments/kernellevel/memorystatisticsglobal.htm

Originally created by @zsnoob on GitHub (Apr 1, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepEP/issues/105 https://github.com/deepseek-ai/DeepEP/issues/21#issuecomment-2764894125 This comment refered that: 'nc encourages NVCC to do more optimizations.' I'm so curious about the hardware side of this architecture-specific characteristic beyond compiler optimization, so I did some summary from docs. Hope others who are also interested in this detail will find the information helpful and get some inspiration for own program design. Though we disable the texture cache's allocation, the behavior how it process access requests may still occur. > it's designed to be a "bandwidth aggregator" which aggregates simultaneous loads request from many threads and tries to stream the results back to the processing units as efficiently as possible, rather than L1 cache that designed for latency. > As the texture cache offers very different coalescing rules over the data cache, using this memory path may improve performance - especially for highly non-uniform memory access patterns. To accelerate memory access, GPUs typically rely on coalescing. However, this optimization comes with certain constraints. The texture cache mechanism can be used to get around coalescing constraints. We have better performance when data access is out of coalesced constraints and have sufficient parallelism. ### Reference: https://docs.nvidia.com/cuda/parallel-thread-execution https://docs.nvidia.com/cuda/cuda-c-best-practices-guide https://docs.nvidia.com/gameworks/content/developertools/desktop/analysis/report/cudaexperiments/kernellevel/memorystatisticsglobal.htm

bsakaguchi closed this issue

2026-10-26 09:54:11 +00:00

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

deepseek-ai/DeepEP#94

No description provided.

Rows
Columns

[GH-ISSUE #105] Some supplementary on ld.nc in PTX model and texture memory for pragmatic guess in README #94

Reference:

[GH-ISSUE #105] Some supplementary on `ld.nc` in PTX model and texture memory for pragmatic guess in README #94