AMD released ROCm 7.13 with Strix Halo optimizations. I benchmarked kyuz0's latest toolbox images against my current ROCm 6.4.4 production baseline to see if upgrading my llama-swap stack is worth it. The answer is complicated.
read more →Getting Unsloth Studio's full training pipeline running on AMD Strix Halo (gfx1151) using pip-packaged ROCm nightlies, no /opt/rocm required. Chat, training, data recipes, and model export all working on Fedora 43.
read more →Google's official Gemma 4 MTP assistant heads bring speculative decoding to MoE models that couldn't benefit before, and nearly quadruple dense model throughput on AMD Strix Halo's bandwidth-limited unified memory.
read more →Multi-Token Prediction turns Qwen 3.6 27B from 6 t/s to 30 t/s on AMD Strix Halo, succeeding where draft models and ngram decoding failed, by using prediction heads baked into the model itself.
read more →Ollama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else's engine. Here's the full history, and why the alternatives are better.
read more →Benchmarking speculative decoding with Gemma 4 E2B as a draft model for Gemma 4 31B on AMD Strix Halo, a bandwidth-bound setup where the optimal draft-max differs from discrete GPUs.
read more →A full-stack web interface that puts the Pi coding agent in the browser, with system-level access, session history, and model switching through a local LiteLLM proxy.
read more →How LiteLLM, llama-swap, and Lemonade Server compose into a unified local inference platform, routing dozens of models across GPU and NPU through a single API endpoint, accessible anywhere via Tailscale and a local reverse proxy.
read more →Setting up AMD's Lemonade Server on Strix Halo to run LLM and Whisper inference on the XDNA 2 NPU, driver builds, architecture decisions, and benchmarks against the integrated GPU.
read more →A Dockerfile and docker-compose setup for running llama.cpp with its Python bindings in a container, because finding a working one shouldn't be this hard.
read more →An all-in-one Docker container that bundles LLMs, embeddings, vision, and TTS into a single unified inference server.
read more →Building a natural language voice controller for the Linux desktop using Qt6, a tiny 1.7B LLM, and a clever vector embedding trick for tool calling.
read more →Building a private, browser-based voice assistant using WebAssembly, Moonshine STT, Piper TTS, and local LLMs.
read more →A text-driven simulation where you interact with a guy named John through natural language, and an LLM determines how his world responds.
read more →