[GH-ISSUE #579] Issue: Incorrect comment in `linear` function #308

New issue

Closed

opened 2026-10-26 21:19:22 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-26 21:19:22 +00:00

Owner

Originally created by @maxiao001 on GitHub (Feb 5, 2025).
Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/579

The comment in the linear function regarding weight.element_size() contains inaccuracies:

    """
    ...

    Notes:
        - If `weight` is quantized (e.g., `element_size() > 1`), a dequantized version 
          is used for computation.
        ...
    """

This comment has two problems:

Incorrect "quantized" condition: element_size() > 1 indicates that the weight tensor is not quantized. Quantized tensors have an element size of 1, while higher precision formats like float32 and bfloat16 have element sizes greater than 1.
Misleading "dequantized version" phrase: The code does not perform any dequantization when weight.element_size() > 1. It directly uses the original weight tensor.

Proposed solution:

Change the comment to accurately reflect the code's logic. A more accurate comment would be:

    """
    ...

    Notes:
        - If `weight` is in a higher precision format (e.g., float32 or bfloat16), then `element_size() > 1`, and the original
          weight tensor is used for computation.
        ...
    """

This revised comment clarifies that no dequantization is performed and the original higher-precision weights are used directly when element_size() > 1.

Originally created by @maxiao001 on GitHub (Feb 5, 2025). Original GitHub issue: https://github.com/deepseek-ai/DeepSeek-V3/issues/579 The comment in the `linear` function regarding `weight.element_size()` contains inaccuracies: ```python """ ... Notes: - If `weight` is quantized (e.g., `element_size() > 1`), a dequantized version is used for computation. ... """ ``` This comment has two problems: 1. **Incorrect "quantized" condition:** `element_size() > 1` indicates that the `weight` tensor is **not** quantized. Quantized tensors have an element size of 1, while higher precision formats like float32 and bfloat16 have element sizes greater than 1. 2. **Misleading "dequantized version" phrase:** The code does not perform any dequantization when `weight.element_size() > 1`. It directly uses the original `weight` tensor. Proposed solution: Change the comment to accurately reflect the code's logic. A more accurate comment would be: ```python """ ... Notes: - If `weight` is in a higher precision format (e.g., float32 or bfloat16), then `element_size() > 1`, and the original weight tensor is used for computation. ... """ ``` This revised comment clarifies that no dequantization is performed and the original higher-precision weights are used directly when `element_size() > 1`.

bsakaguchi

2026-10-26 21:19:22 +00:00

closed this issue
added the
stale

closed-as-stale
labels

bsakaguchi commented

2026-10-26 21:19:23 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Mar 8, 2025):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

@github-actions[bot] commented on GitHub (Mar 8, 2025): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!

bsakaguchi commented

2026-10-26 21:19:23 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Mar 23, 2025):

false

@github-actions[bot] commented on GitHub (Mar 23, 2025): false