Qwen 3.8 27B on Strix Halo: 18 t/s With MTP, Nearly 2x the Baseline

Qwen just dropped Qwen 3.8, a 27B dense model with vision, hybrid thinking, 256K context, and reasoning effort controls. It’s the successor to Qwen 3.6, which I’ve been running with MTP at 30 t/s as my quality-focused local model. The obvious question: does MTP work on 3.8, and what’s the fastest I can push it?

The answer is 18 t/s with MTP, up from a 9.6 t/s baseline. Not quite the 4.8x jump that Qwen 3.6 saw, but the baseline is already faster here (9.6 vs 6.2 t/s), and the absolute speed is comfortable for interactive use.

#The Setup

Component	Specification
CPU/GPU	AMD Ryzen AI MAX+ 395 / Radeon 8060S (Strix Halo)
RAM	128 GB LPDDR5X unified memory (~218 GB/s)
OS	Fedora 44
Backend	Vulkan (Mesa RADV, llama-vulkan-radv toolbox)
Model (base)	Qwen3.8-27B-Q6_K.gguf (22 GB)
Model (MTP)	Qwen3.8-27B-MTP-Q6_K.gguf (21 GB)
Chat template	froggeric/Qwen-Fixed-Chat-Templates v22

Unlike my Qwen 3.6 MTP experiment, which required building am17an’s fork from source on the host, Qwen 3.8 MTP works out of the box with the updated llama-vulkan-radv toolbox. No custom binary needed. The toolbox already has draft-mtp support.

One caveat: the ROCm 7.2.4 toolbox cannot load Qwen 3.8 or Muse Glimmer yet (unknown model architecture: muse-glimmer / works but requires the newer Vulkan path for Qwen 3.8 MTP). Both the llama-vulkan-radv and llama-vulkan-amdvlk toolboxes handle them fine.

#Toolbox Comparison: RadV vs AMDVLK

I benchmarked both Vulkan ICDs across all configurations. Each variant was tested with llama-swap restarted between runs to avoid memory contention, 1 warmup + 3 measured runs with a fixed prompt at max_tokens=256.

Variant	Toolbox	Mean decode tok/s	Mean prompt tok/s	Mean elapsed (s)
Base (no MTP)	radv	9.59	26.01	22.27
Base (no MTP)	amdvlk	9.59	16.73	28.03
MTP n2	radv	18.01	23.90	14.11
MTP n2	amdvlk	17.84	4.77	15.30
MTP n5	radv	15.16	23.43	14.19
MTP n5	amdvlk	16.48	4.74	14.80

RadV wins across the board. Decode speeds are similar between the two drivers, but prompt processing is dramatically faster on radv (24 tok/s vs 5 tok/s), which translates directly to lower time-to-first-token and better total latency. The only category where amdvlk is competitive is raw decode throughput on MTP n5, where it’s slightly ahead of radv’s n5 (16.5 vs 15.2 t/s), but still behind radv’s n2 result.

#MTP: n2 vs n5

This is the opposite of what I saw with Qwen 3.6, where n-max 5 was consistently faster than n-max 2. On Qwen 3.8 with Q6_K weights:

draft-n-max	Decode tok/s (radv)	vs Baseline
2	18.01	+88%
5	15.16	+58%

The n2 config outperforms n5 by about 19%. My working theory: the Q6_K quant combined with the larger MTP head in 3.8 means speculating 5 tokens generates more rejected candidates than accepted ones at this quantization level. The overhead of verifying and discarding those extra drafts eats into the throughput gain. With n2, almost everything proposed gets accepted, so the speculative overhead stays minimal.

#Preserve Thinking and Cache Behavior

Qwen 3.8 introduces reasoning_effort controls alongside the existing thinking toggles. The key configuration for efficient multi-turn cache reuse:

--chat-template-kwargs "{\"preserve_thinking\":true}"

This keeps past <think> blocks in the conversation history so the KV prefix cache stays valid across turns. Without it, each turn would invalidate the entire cache because the token sequence changes when thinking blocks are stripped.

I’m using froggeric’s fixed template v22, which adds Qwen 3.8 support while fixing several regressions in the official template:

Official 3.8 throws an exception if you pass enable_thinking=false. The fixed template restores fast mode.
Official 3.8 injects duplicate blank <think></think> blocks into chat history. The fixed template extracts reasoning cleanly.
Official 3.8 crashes with standard OpenAI-format string tool arguments. The fixed template handles both dict and JSON string arguments.

The --reasoning-preserve flag in recent llama.cpp builds also works as an alternative to template-level preservation, but passing it through --chat-template-kwargs keeps the behavior explicit in the config.

#The Final Config

The winning profile as deployed in my llama-swap config:

"qwen3.8-27b":
  cmd: >
    /usr/bin/toolbox run --container llama-vulkan-radv llama-server
    --host 0.0.0.0 --port ${PORT}
    -m ~/Secondary/Models/Qwen3.8-27B-MTP-Q6_K.gguf
    -fa on -ngl 999 -b 2048 -ub 2048 --cache-reuse 1 --no-mmap
    --spec-type draft-mtp --spec-draft-n-max 2
    -np 1 --ctx-size 131072
    --cache-type-k q8_0 --cache-type-v q8_0
    --no-context-shift
    --jinja --chat-template-file ~/llama-swap/templates/chat_template.jinja
    --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
    --presence-penalty 0.0 --repeat-penalty 1.0
    --chat-template-kwargs "{\"preserve_thinking\":true}"
  aliases:
    - "openai/qwen3.8-27b"
    - "local/Qwen3.8-27B"

Key choices:

vulkan-radv toolbox, not amdvlk or ROCm
MTP with draft-n-max 2, not 5
Q6_K quant (22 GB including MTP heads), good quality/speed balance on 128 GB unified memory
q8_0 KV cache for quality, since we have the memory budget
preserve_thinking enabled for multi-turn KV prefix cache hits
froggeric’s v22 template for tool calling and thinking mode compatibility

#Comparison to My Stack

Model	Method	Decode tok/s	Use case
Qwen 3.6 35B-A3B (MoE)	Baseline	~33	Daily driver, fast non-thinking
Qwen 3.6 27B	MTP Q4_K_M	~30	Dense quality, max speed
Qwen 3.8 27B	MTP Q6_K	~18	Dense quality, latest model gen
Qwen 3.6 27B	MTP Q8_0	~19	Dense quality, max fidelity

Qwen 3.8 with MTP at 18 t/s slots in right alongside the Q8_0 tier of Qwen 3.6 MTP. It’s not faster, but it’s a newer model generation with better benchmarks across coding, agentic, and reasoning tasks. The tradeoff is clear: same speed, better model.

#Bottom Line

Qwen 3.8 27B with MTP on Strix Halo delivers 18 t/s, which is comfortable for interactive coding and agentic workflows. The setup is simpler than Qwen 3.6 MTP was: no custom forks, no host builds, just a standard Vulkan toolbox with the right flags. If you’re on Strix Halo or similar bandwidth-constrained hardware, the key decisions are:

Use llama-vulkan-radv, not amdvlk or ROCm (for Qwen 3.8)
Use draft-n-max 2, not 5 (at Q6_K)
Use froggeric’s fixed template for tool calling and thinking mode
Enable preserve_thinking for cache efficiency