kvcache-ai/sglang

Fork 0

mirror of https://github.com/kvcache-ai/sglang.git synced 2026-11-05 01:07:39 +00:00

[PR #92] [MERGED] fix(openai): support DSV4 reasoning in Responses API #92

New issue

Closed

opened 2026-10-29 21:07:08 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-29 21:07:08 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/kvcache-ai/sglang/pull/92
Author: @yyj6666667
Created: 9/9/2026
Status: ✅ Merged
Merged: 9/9/2026
Merged by: @yyj6666667

Base: main ← Head: fix/dsv4-reasoning-responses-codex

📝 Commits (2)

2819389 fix(openai): support DSV4 reasoning in Responses API
db87597 style: sort DSV4 OpenAI imports

📊 Changes

7 files changed (+1608 additions, -153 deletions)

View changed files

➕ python/sglang/srt/entrypoints/openai/chat_encoding.py (+107 -0)
📝 python/sglang/srt/entrypoints/openai/encoding_dsv4.py (+44 -12)
📝 python/sglang/srt/entrypoints/openai/protocol.py (+76 -30)
📝 python/sglang/srt/entrypoints/openai/serving_chat.py (+135 -31)
📝 python/sglang/srt/entrypoints/openai/serving_responses.py (+846 -78)
📝 python/sglang/srt/environ.py (+3 -2)
➕ test/registered/unit/entrypoints/openai/test_dsv4_reasoning_responses.py (+397 -0)

📄 Description

Summary

mirror the DeepSeek-V4-Flash-0731 low / high / max prompt mapping while preserving the older preview checkpoint mapping
resolve Chat Completions reasoning_effort and Responses reasoning.effort through one DSV4 policy; omitted/null defaults to thinking + max, none disables thinking, and unsupported strings warn and fall back to thinking + max
keep chat_template_kwargs and the old environment variable as compatibility-only fallbacks
add non-Harmony Responses streaming, official Responses usage fields, reasoning summaries, and top-level function-tool round trips required by Codex
accept namespace, web_search, and reasoning.encrypted_content request shapes for wire compatibility without claiming to execute unsupported tools or synthesize encrypted reasoning

This fixes the one-level-off encoder mapping described in sgl-project/sglang#33185, and also fixes the intermediate Responses API path where Codex's standard effort field previously did not reach the DSV4 encoder.

DSV4 behavior

Input	Effective behavior
omitted / `null`	thinking + `max`
`low`	thinking + `low`
`high`	thinking + `high`
`max`	thinking + `max`
`none`	thinking disabled
`minimal` / `medium` / `xhigh` / other strings	warning + thinking + `max`

Effort selects a model prompt prefix. It does not alter the token budget or kernel scheduler.

Compatibility boundary

Chat Completions: top-level reasoning_effort
Responses/Codex: reasoning: {"effort": ..., "summary": ...}
top-level Responses function tools are exposed to DSV4 and round-trip through function_call_output
namespace and web_search objects are accepted but deliberately not exposed or executed in the non-Harmony path

Validation

targeted unit suite: 10/10 passed in the source-wheel validation image; the only later source change was import sorting
Black check, Python byte compilation, and git diff --check: passed
qj5090, 2 x RTX 5090, Chat API: all 10 effort cases returned HTTP 200 with the expected thinking state and fallback warnings
qj5090 Responses API: non-stream response passed the official OpenAI SDK schema and returned input_tokens / output_tokens usage fields
Codex CLI 0.153.4 end to end: captured reasoning.effort=max, generated an exec_command call, consumed its function_call_output, returned the marker exactly, and exited 0

The qj5090 run used an immutable image rebuilt with a wheel from the functional SGLang commit. The subsequent PR-head change only sorted imports. The already-running production container was not modified.

CI note

The heavyweight test workflows are gated on the maintainer-controlled run-ci label. The automatic lint workflow runs pre-commit over the entire fork and currently fails on existing files outside this PR (including 79 Black rewrites, existing shebang modes, and unrelated Ruff errors). After the import-only fix, none of this PR's seven files appears in the formatter rewrite list; their targeted isort, Black, Ruff F401/F821, AST/bytecode, codespell, and whitespace checks pass.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/kvcache-ai/sglang/pull/92 **Author:** [@yyj6666667](https://github.com/yyj6666667) **Created:** 9/9/2026 **Status:** ✅ Merged **Merged:** 9/9/2026 **Merged by:** [@yyj6666667](https://github.com/yyj6666667) **Base:** `main` ← **Head:** `fix/dsv4-reasoning-responses-codex` --- ### 📝 Commits (2) - [`2819389`](https://github.com/kvcache-ai/sglang/commit/2819389c09f18c5423369cc1e28dad7429829d96) fix(openai): support DSV4 reasoning in Responses API - [`db87597`](https://github.com/kvcache-ai/sglang/commit/db8759715183600aebd947ddd4654a73795ad7a3) style: sort DSV4 OpenAI imports ### 📊 Changes **7 files changed** (+1608 additions, -153 deletions) <details> <summary>View changed files</summary> ➕ `python/sglang/srt/entrypoints/openai/chat_encoding.py` (+107 -0) 📝 `python/sglang/srt/entrypoints/openai/encoding_dsv4.py` (+44 -12) 📝 `python/sglang/srt/entrypoints/openai/protocol.py` (+76 -30) 📝 `python/sglang/srt/entrypoints/openai/serving_chat.py` (+135 -31) 📝 `python/sglang/srt/entrypoints/openai/serving_responses.py` (+846 -78) 📝 `python/sglang/srt/environ.py` (+3 -2) ➕ `test/registered/unit/entrypoints/openai/test_dsv4_reasoning_responses.py` (+397 -0) </details> ### 📄 Description ## Summary - mirror the DeepSeek-V4-Flash-0731 `low` / `high` / `max` prompt mapping while preserving the older preview checkpoint mapping - resolve Chat Completions `reasoning_effort` and Responses `reasoning.effort` through one DSV4 policy; omitted/null defaults to thinking + `max`, `none` disables thinking, and unsupported strings warn and fall back to thinking + `max` - keep `chat_template_kwargs` and the old environment variable as compatibility-only fallbacks - add non-Harmony Responses streaming, official Responses usage fields, reasoning summaries, and top-level function-tool round trips required by Codex - accept `namespace`, `web_search`, and `reasoning.encrypted_content` request shapes for wire compatibility without claiming to execute unsupported tools or synthesize encrypted reasoning This fixes the one-level-off encoder mapping described in [sgl-project/sglang#33185](https://github.com/sgl-project/sglang/issues/33185), and also fixes the intermediate Responses API path where Codex's standard effort field previously did not reach the DSV4 encoder. ## DSV4 behavior | Input | Effective behavior | | --- | --- | | omitted / `null` | thinking + `max` | | `low` | thinking + `low` | | `high` | thinking + `high` | | `max` | thinking + `max` | | `none` | thinking disabled | | `minimal` / `medium` / `xhigh` / other strings | warning + thinking + `max` | Effort selects a model prompt prefix. It does not alter the token budget or kernel scheduler. ## Compatibility boundary - Chat Completions: top-level `reasoning_effort` - Responses/Codex: `reasoning: {"effort": ..., "summary": ...}` - top-level Responses `function` tools are exposed to DSV4 and round-trip through `function_call_output` - `namespace` and `web_search` objects are accepted but deliberately not exposed or executed in the non-Harmony path ## Validation - targeted unit suite: 10/10 passed in the source-wheel validation image; the only later source change was import sorting - Black check, Python byte compilation, and `git diff --check`: passed - qj5090, 2 x RTX 5090, Chat API: all 10 effort cases returned HTTP 200 with the expected thinking state and fallback warnings - qj5090 Responses API: non-stream response passed the official OpenAI SDK schema and returned `input_tokens` / `output_tokens` usage fields - Codex CLI 0.153.4 end to end: captured `reasoning.effort=max`, generated an `exec_command` call, consumed its `function_call_output`, returned the marker exactly, and exited 0 The qj5090 run used an immutable image rebuilt with a wheel from the functional SGLang commit. The subsequent PR-head change only sorted imports. The already-running production container was not modified. ## CI note The heavyweight test workflows are gated on the maintainer-controlled `run-ci` label. The automatic lint workflow runs pre-commit over the entire fork and currently fails on existing files outside this PR (including 79 Black rewrites, existing shebang modes, and unrelated Ruff errors). After the import-only fix, none of this PR's seven files appears in the formatter rewrite list; their targeted isort, Black, Ruff `F401/F821`, AST/bytecode, codespell, and whitespace checks pass. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-10-29 21:07:08 +00:00

closed this issue
added the
pull-request
label

No labels

pull-request

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kvcache-ai/sglang#92

No description provided.

Rows
Columns