convert.py is used to convert huggingface checkpoints into model parallel checkpoints required by the demo, so that each rank only needs to load the corresponding checkpoint when running the demo

@GeeeekExplorer commented on GitHub (Dec 31, 2024): `convert.py` is used to convert huggingface checkpoints into model parallel checkpoints required by the demo, so that each rank only needs to load the corresponding checkpoint when running the demo

bsakaguchi commented

2026-10-26 21:11:08 +00:00

Author

Owner

@KKwanhee commented on GitHub (Feb 3, 2025):

Why was layer61 (corresponding to nextn_predict_layers) excluded in convert.py when generating the safetensor? Doesn't that mean layer61 won't be used during inference?

@KKwanhee commented on GitHub (Feb 3, 2025): Why was layer61 (corresponding to `nextn_predict_layers`) excluded in `convert.py` when generating the safetensor? Doesn't that mean `layer61` won't be used during inference?

bsakaguchi commented

2026-10-26 21:11:08 +00:00

Author

Owner

@GeeeekExplorer commented on GitHub (Feb 5, 2025):

Why was layer61 (corresponding to nextn_predict_layers) excluded in convert.py when generating the safetensor? Doesn't that mean layer61 won't be used during inference?

Yes, The infer demo currently does not support MTP which uses layer 61.

@GeeeekExplorer commented on GitHub (Feb 5, 2025): > Why was layer61 (corresponding to `nextn_predict_layers`) excluded in `convert.py` when generating the safetensor? Doesn't that mean `layer61` won't be used during inference? Yes, The infer demo currently does not support MTP which uses layer 61.

bsakaguchi commented

2026-10-26 21:11:09 +00:00

Author

Owner

@Huixxi commented on GitHub (Feb 20, 2025):

But --model-parallel 32 seems also do not supported?

Why was layer61 (corresponding to nextn_predict_layers) excluded in convert.py when generating the safetensor? Doesn't that mean layer61 won't be used during inference?

Yes, The infer demo currently does not support MTP which uses layer 61.

@Huixxi commented on GitHub (Feb 20, 2025): But --model-parallel 32 seems also do not supported? > > Why was layer61 (corresponding to `nextn_predict_layers`) excluded in `convert.py` when generating the safetensor? Doesn't that mean `layer61` won't be used during inference? > > Yes, The infer demo currently does not support MTP which uses layer 61.