Write-ups of what I built and how it went.
Qwen 3.8 27B hits 18 t/s via MTP speculative decoding on Strix Halo, up from 9.6 t/s baseline. The winning config uses vulkan-radv, draft-n-max 2, and froggeric's fixed template for proper cache preservation.
read more →Hy3's 299B MoE runs at 10 tok/s on 128 GB Strix Halo in baseline mode, but the MTP speculative head that gives CUDA users a +27-37% boost can't allocate on Vulkan unified memory. Here's what worked, what didn't, and why.
read more →Colibri streams GLM-5.2's 19,456 experts from disk to run a 744-billion-parameter MoE model on consumer hardware. Here's what happens when you point it at 128 GB of unified memory and a PCIe 4.0 NVMe.
read more →