Skip to content

[doc] fix qwen35 gkd deploy - #9718

Merged
hjh0119 merged 1 commit into
modelscope:mainfrom
hjh0119:fix-doc-0710
Jul 10, 2026
Merged

hjh0119 merged 1 commit into
modelscope:mainfrom
hjh0119:fix-doc-0710

Conversation

@hjh0119

@hjh0119 hjh0119 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the documentation for Qwen3.5 best practices (both Chinese and English versions) to use swift deploy instead of vllm serve for launching the teacher server during GKD training. The review feedback points out that the --max_logprobs parameter should be prefixed with --vllm_ (i.e., --vllm_max_logprobs) to be correctly parsed by swift deploy. Additionally, it recommends correcting the definition of GKD to "Generalized Knowledge Distillation" in the English documentation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

--vllm_max_model_len 10240 \
--gpu-memory-utilization 0.8 \
--max-logprobs 64
--max_logprobs 64

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In swift deploy, vLLM-specific arguments should be prefixed with --vllm_ to be correctly parsed and passed to the vLLM engine. Please use --vllm_max_logprobs instead of --max_logprobs.

Suggested change
--max_logprobs 64
--vllm_max_logprobs 64

### GKD

LoRA training with GKD (Guided Knowledge Distillation), using Qwen3.5-9B as the teacher model. First, launch the teacher server with vLLM (alternatively, use the `--teacher_model` parameter to load the model directly):
LoRA training with GKD (General Knowledge Distillation), using Qwen3.5-9B as the teacher model. First, launch the teacher server with `swift deploy` (alternatively, use the `--teacher_model` parameter to load the model directly):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

GKD stands for Generalized Knowledge Distillation (from the GKD paper) rather than "General Knowledge Distillation". Let's correct this terminology for accuracy.

Suggested change
LoRA training with GKD (General Knowledge Distillation), using Qwen3.5-9B as the teacher model. First, launch the teacher server with `swift deploy` (alternatively, use the `--teacher_model` parameter to load the model directly):
LoRA training with GKD (Generalized Knowledge Distillation), using Qwen3.5-9B as the teacher model. First, launch the teacher server with swift deploy (alternatively, use the --teacher_model parameter to load the model directly):

--vllm_max_model_len 10240 \
--gpu-memory-utilization 0.8 \
--max-logprobs 64
--max_logprobs 64

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In swift deploy, vLLM-specific arguments should be prefixed with --vllm_ to be correctly parsed and passed to the vLLM engine. Please use --vllm_max_logprobs instead of --max_logprobs.

Suggested change
--max_logprobs 64
--vllm_max_logprobs 64

@hjh0119
hjh0119 merged commit 0a4f08e into modelscope:main Jul 10, 2026
1 check passed
@hjh0119
hjh0119 deleted the fix-doc-0710 branch July 10, 2026 06:01
Jintao-Huang pushed a commit that referenced this pull request Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants