[GH-ISSUE #78] proposal: improve the topology metadata #44

New issue

Closed

opened 2026-10-29 21:08:19 +00:00 by bsakaguchi · 4 comments

bsakaguchi commented

2026-10-29 21:08:19 +00:00

Owner

Originally created by @doujiang24 on GitHub (Jan 13, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/78

Now, we store the simple priority_matrix in metadata, that including the prefer/available nics for cpu numas/gpus.

I'm here proposal a richer/original topolopy matrix metadata, similar to NCCL_TOPO_FILE, which contains more useful info, including if gdr supported, nvlink, nvswitch, and etc.

After that, we do not need to store the protocol in metadata, instead, we could choose a proper protocol base on the topology.

Here is a simple sample topo from nccl:

<system version="1">
  <cpu host_hash="0xb52dbdea56f9ca26" numaid="0" affinity="00000000,ffffffff,00000000,ffffffff" arch="x86_64" vendor="GenuineIntel" familyid="6" modelid="106">
    <pci busid="0000:4d:00.0" class="0x060400" vendor="0x1000" device="0xc010" subsystem_vendor="0x1000" subsystem_device="0xa096" link_speed="" link_width="0">
      <pci busid="0000:56:00.0" class="0x060400" vendor="0x1000" device="0xc010" subsystem_vendor="0x1000" subsystem_device="0xa096" link_speed="" link_width="0">
        <pci busid="0000:58:00.0" class="0x060400" vendor="0x1000" device="0xc010" subsystem_vendor="0x10de" subsystem_device="0x13b8" link_speed="" link_width="0">
          <pci busid="0000:5a:00.0" class="0x030200" vendor="0x10de" device="0x20b2" subsystem_vendor="0x10de" subsystem_device="0x1463" link_speed="" link_width="0">
            <gpu dev="1" sm="80" rank="1" gdr="1">
              <nvlink target="0000:f0:00.0" count="2" tclass="0x068000"/>
              <nvlink target="0000:ed:00.0" count="2" tclass="0x068000"/>
            </gpu>
          </pci>
        </pci>
        <pci busid="0000:5b:00.0" class="0x020000" vendor="0x15b3" device="0x101d" subsystem_vendor="0x15b3" subsystem_device="0x0064" link_speed="" link_width="0">
          <nic>
            <net name="mlx5_bond_0" dev="0" speed="100000" port="1" latency="0.000000" guid="0xce0fda0003ebc008" maxconn="262144" gdr="1"/>
          </nic>
        </pci>
      </pci>
    </pci>
  </cpu>
</system>

Originally created by @doujiang24 on GitHub (Jan 13, 2025). Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/78 Now, we store the simple `priority_matrix` in metadata, that including the prefer/available nics for cpu numas/gpus. I'm here proposal a richer/original topolopy matrix metadata, similar to [NCCL_TOPO_FILE](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/env.html), which contains more useful info, including if gdr supported, nvlink, nvswitch, and etc. After that, we do not need to store the protocol in metadata, instead, we could choose a proper protocol base on the topology. Here is a simple sample topo from nccl: ``` <system version="1"> <cpu host_hash="0xb52dbdea56f9ca26" numaid="0" affinity="00000000,ffffffff,00000000,ffffffff" arch="x86_64" vendor="GenuineIntel" familyid="6" modelid="106"> <pci busid="0000:4d:00.0" class="0x060400" vendor="0x1000" device="0xc010" subsystem_vendor="0x1000" subsystem_device="0xa096" link_speed="" link_width="0"> <pci busid="0000:56:00.0" class="0x060400" vendor="0x1000" device="0xc010" subsystem_vendor="0x1000" subsystem_device="0xa096" link_speed="" link_width="0"> <pci busid="0000:58:00.0" class="0x060400" vendor="0x1000" device="0xc010" subsystem_vendor="0x10de" subsystem_device="0x13b8" link_speed="" link_width="0"> <pci busid="0000:5a:00.0" class="0x030200" vendor="0x10de" device="0x20b2" subsystem_vendor="0x10de" subsystem_device="0x1463" link_speed="" link_width="0"> <gpu dev="1" sm="80" rank="1" gdr="1"> <nvlink target="0000:f0:00.0" count="2" tclass="0x068000"/> <nvlink target="0000:ed:00.0" count="2" tclass="0x068000"/> </gpu> </pci> </pci> <pci busid="0000:5b:00.0" class="0x020000" vendor="0x15b3" device="0x101d" subsystem_vendor="0x15b3" subsystem_device="0x0064" link_speed="" link_width="0"> <nic> <net name="mlx5_bond_0" dev="0" speed="100000" port="1" latency="0.000000" guid="0xce0fda0003ebc008" maxconn="262144" gdr="1"/> </nic> </pci> </pci> </pci> </cpu> </system> ```

bsakaguchi

2026-10-29 21:08:19 +00:00

closed this issue
added the
stale

auto-closed
labels

bsakaguchi commented

2026-10-29 21:08:20 +00:00

Author

Owner

@alogfans commented on GitHub (Jan 13, 2025):

The proposed topology provides more information than the current version, i.e., supported mode / max bandwidth per RNIC device.
It is more rely on auto-detection because its hard to manually input.
The topology file should be shared among the cluster, i.e., it is also stored in global metadata storage.
The global metadata storage also includes parameters of RDMA connection (e.g., GID, QPNum, ...), which the proposed topology file does not include.

@alogfans commented on GitHub (Jan 13, 2025): - The proposed topology provides more information than the current version, i.e., supported mode / max bandwidth per RNIC device. - It is more rely on auto-detection because its hard to manually input. - The topology file should be shared among the cluster, i.e., it is also stored in global metadata storage. - The global metadata storage also includes parameters of RDMA connection (e.g., GID, QPNum, ...), which the proposed topology file does not include.

bsakaguchi commented

2026-10-29 21:08:21 +00:00

Author

Owner

@doujiang24 commented on GitHub (Jan 13, 2025):

Yep, totally agreed, I'll figure out a json schema before creating a PR, which including GID/LID, ...

@doujiang24 commented on GitHub (Jan 13, 2025): Yep, totally agreed, I'll figure out a json schema before creating a PR, which including GID/LID, ...

bsakaguchi commented

2026-10-29 21:08:21 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 23, 2026):

This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

@github-actions[bot] commented on GitHub (Jul 23, 2026): This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.

bsakaguchi referenced this issue

2026-10-29 21:08:21 +00:00

[GH-ISSUE #76] Libfabric transport layer support #43

bsakaguchi commented

2026-10-29 21:08:21 +00:00

Author

Owner

@github-actions[bot] commented on GitHub (Jul 30, 2026):

Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

@github-actions[bot] commented on GitHub (Jul 30, 2026): Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.

bsakaguchi referenced this issue

2026-10-29 21:09:12 +00:00

[GH-ISSUE #215] Request for Good First Issue about OSPP #95