Enable native ModelOpt quantization support (3/3) by Edwardf0t1 · Pull Request #2 · Edwardf0t1/sglang

Edwardf0t1 · 2025-09-09T04:50:01Z

This PR only shows the diff between the 2nd and 3rd PR for a three-part series to enable native ModelOpt quantization in SGLang

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

…#11449)

…gl-project#11601)

…nager` utility function (sgl-project#11586)

Signed-off-by: Alex Chi Z <iskyzh@gmail.com>

…` and add `minijinja-contrib` (sgl-project#11882)

Co-authored-by: Simo Lin <linsimo.mark@gmail.com>

Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>

…ogs (sgl-project#11886)

…ject#11887)

Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com> Co-authored-by: 羽癫 <yudian.zy@antgroup.com> Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>

Co-authored-by: guangyey <guangye.yu@intel.com> Co-authored-by: DiweiSun <105627594+DiweiSun@users.noreply.github.com>

…sgl-project#11664)

Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>

Signed-off-by: xintin <gaurav.verma@amd.com>

…1487) Co-authored-by: Jianwei Dong <1913953267@qq.com>

…1570)

sgl-project#6572] (sgl-project#11416) Signed-off-by: ybyang <ybyang7@iflytek.com> Co-authored-by: YorkSu <york_su@qq.com>

Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>

…#11902) Signed-off-by: Shangming Cai <csmthu@gmail.com>

)

Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>

…e_intermediate_size` / `weight_block_size_n` (sgl-project#11702) Signed-off-by: Kai-Hsun Chen <khchen@x.ai>

Signed-off-by: Shangming Cai <csmthu@gmail.com>

Signed-off-by: Serge Panev <spanev@nvidia.com>

…gl-project#11876)

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch from 2674259 to aed7dd2 Compare September 12, 2025 23:32

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch 2 times, most recently from 19fcedb to 95fc54b Compare September 13, 2025 01:48

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch from f074579 to c13b457 Compare September 18, 2025 06:21

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch from 95fc54b to d25e5d1 Compare September 23, 2025 08:18

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch from 1c16530 to 54524e2 Compare September 25, 2025 23:19

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch from d25e5d1 to a9e4353 Compare September 26, 2025 06:25

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch from c118561 to e75fbf3 Compare September 29, 2025 23:56

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch from b66d1dc to c5181b3 Compare September 30, 2025 00:34

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch from e75fbf3 to 5c1587f Compare September 30, 2025 05:26

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch from c5181b3 to 15dd13e Compare September 30, 2025 05:34

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch 2 times, most recently from 7134aa5 to fe3ee4e Compare October 7, 2025 08:23

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch 2 times, most recently from 9c2eaac to 6c34fd9 Compare October 9, 2025 07:56

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch from fcfd22b to ff8cb61 Compare October 9, 2025 08:11

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch 2 times, most recently from f1dd65e to 6769000 Compare October 9, 2025 20:11

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-2 branch from 40fefb3 to 9bc99e7 Compare October 11, 2025 03:13

Edwardf0t1 force-pushed the zhiyu/modelopt-sglang-api-3 branch 3 times, most recently from 7b27705 to 456a3f9 Compare October 14, 2025 08:24

slin1237 and others added 8 commits October 14, 2025 01:42

[router] allow router launch server to use grpc mode (sgl-project#11600)

da7fac1

[Docs] [Router]: Update sg-router doc on circuit breaker (sgl-project…

642fa96

…#11449)

[router] when given both local tokenizer and chat template, log all (s…

a04efc4

…gl-project#11601)

[AMD CI] Add image and weights caching. (sgl-project#11593)

1d08653

Update release-docker-dev.yml (sgl-project#11603)

9767a1e

Optimize Triton Draft Backend (sgl-project#11556)

5a33c3a

Refactor spec decoding metrics calculation into separate `TokenizerMa…

817e46f

…nager` utility function (sgl-project#11586)

make radix cache deterministic (sgl-project#10721)

dc965db

Signed-off-by: Alex Chi Z <iskyzh@gmail.com>

CatherineSue and others added 30 commits October 20, 2025 18:03

[router][grpc] Remove continue_final_message in `ChatTemplateParams…

e69094d

…` and add `minijinja-contrib` (sgl-project#11882)

fix(sql-router): fix conflict port in test (sgl-project#11826)

28ddfb3

Co-authored-by: Simo Lin <linsimo.mark@gmail.com>

add the missing modelopt_config.py

956a0fb

Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>

[router] clean up workflow logs to debug for implementation details l…

1111030

…ogs (sgl-project#11886)

[code move] move pp into a separate mixin (sgl-project#11838)

01f14a7

[router][grpc] Fix wram-up random token ids for small models (sgl-pro…

9c0b1eb

…ject#11887)

Revise MRotaryEmbedding's forward (sgl-project#11859)

74de76c

Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com> Co-authored-by: 羽癫 <yudian.zy@antgroup.com> Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>

piecewise cuda graph support qwen3-moe (sgl-project#11845)

8374a96

Fix RotaryEmbedding for fp32 input (sgl-project#11843)

fb6cc7b

Init attention backend for Intel XPU (sgl-project#10656)

b113c72

Co-authored-by: guangyey <guangye.yu@intel.com> Co-authored-by: DiweiSun <105627594+DiweiSun@users.noreply.github.com>

Use trtllm_mla decode kernel for draft extend in speculative decoding (…

d9a20fd

…sgl-project#11664)

update example script and doc

725462a

Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>

[router] release router 0.2.1 (sgl-project#11885)

8a801ee

[AMD] Update wave-lang to 3.8.0 (sgl-project#11878)

6f9b66b

Signed-off-by: xintin <gaurav.verma@amd.com>

init support for KTransformers Heterogeneous Computing (sgl-project#1…

7e6191c

…1487) Co-authored-by: Jianwei Dong <1913953267@qq.com>

[FEATURE] Add OpenAI-Compatible LoRA Adapter Selection (sgl-project#1…

852c057

…1570)

[fix] fix ci uv install dependency (sgl-project#11895)

c1e1600

Support Thinking Budget (via custom_logit_processor for OpenAI API) [Fix

dbb16be

sgl-project#6572] (sgl-project#11416) Signed-off-by: ybyang <ybyang7@iflytek.com> Co-authored-by: YorkSu <york_su@qq.com>

Simplify multi-tokenizer (sgl-project#11295)

260fe75

Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>

[CI] disable glm4.1v and fix the flashinfer installation (sgl-project…

05d3667

…#11902) Signed-off-by: Shangming Cai <csmthu@gmail.com>

vlm: enforce pybase64 for image and str encode/decode (sgl-project#10700

d0a64c7

)

[smol] [perf] Inverse perm improvement (sgl-project#11482)

e8640ee

Signed-off-by: vincentzed <207368749+vincentzed@users.noreply.github.com>

[quantization][MoE] fix the check for tp_size / moe_ep_size / `mo…

c61b0b2

…e_intermediate_size` / `weight_block_size_n` (sgl-project#11702) Signed-off-by: Kai-Hsun Chen <khchen@x.ai>

[CI] Fix b200 flashinfer installation (sgl-project#11915)

f3cd5d2

Signed-off-by: Shangming Cai <csmthu@gmail.com>

Fix flush cache API for spec v2 (sgl-project#11918)

97710cc

[NVIDIA] Add new SMs support for Spark & Thor (sgl-project#11287)

2b1da82

Signed-off-by: Serge Panev <spanev@nvidia.com>

Update sgl-kernel and remove fast hadamard depedency (sgl-project#11844)

ebff4ee

Rename flashmla kernel options of nsa backend for better readability (s…

ef4a809

…gl-project#11876)

chore: upgrade flashinfer 0.4.1 (sgl-project#11933)

9792b9d

Merge branch 'main' into zhiyu/modelopt-sglang-api-3

f9a8e17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Enable native ModelOpt quantization support (3/3)#2

Enable native ModelOpt quantization support (3/3)#2
Edwardf0t1 wants to merge 348 commits intozhiyu/modelopt-sglang-api-2from
zhiyu/modelopt-sglang-api-3

Edwardf0t1 commented Sep 9, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Comments

Conversation

Edwardf0t1 commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Edwardf0t1 commented Sep 9, 2025 •

edited

Loading