Arc B580 Stuck at 13 tok/s? IPEX-LLM Unlocks 70

Run IPEX-LLM through Docker on Linux, not Vulkan, to get ~70 tok/s on Mistral 7B with the Arc B580. The generic Vulkan backend tops out at 13–15 tok/s because it can't access Intel's XMX matrix units. IPEX-LLM can. The Docker setup takes 15 minutes once and delivers a permanent 5× speedup. Use Vulkan only for quick tests where setup overhead isn't worth it.

Why Your Arc B580 Feels Slow — The Vulkan Trap

The generic Vulkan backend on Arc B580 yields 13–15 tok/s on Mistral 7B Q4_K_M, the same generic compute path AMD and NVIDIA cards fall back to, leaving Arc's XMX matrix units idle. You bought this card because headlines promised 70 tok/s for $250. Now you're staring at a terminal wondering if the hardware is defective. It's not. The Arc B580 packs 20 Xe-cores with dedicated XMX matrix acceleration hardware. Vulkan can't access any of it. Vulkan was designed as a lowest-common-denominator graphics API, and when llama.cpp or Ollama defaults to --gpu VULKAN, they're asking the GPU to run inference through generic compute shaders that treat your B580 like a 2018 integrated graphics part. Matrix multiplications that could hit XMX at 1024 INT8 ops/clock instead plod through the standard ALU path. The gap is stark. At 13 tok/s, you wait ~77 ms between tokens, barely interactive for chat. You wait for the cursor to blink. Then you Google "Arc B580 slow llama.cpp" and find forum threads full of people confirming the same numbers with no explanation.

The mismatch is architectural. The software asks for generic compute while the hardware delivers matrix acceleration. The B580's Xe2-HPG architecture shares its DNA with the A770, including XMX units Intel added for AI workloads. Most tutorials recommend a path that bypasses them. Users report watching clinfo show a healthy GPU, Ollama detect "Intel Arc B580" in its startup log, and still getting 14 tok/s — because the detection only means Vulkan found a device, not that it's using the right backend. The GPU detection looks correct, the setup was trivial, and the performance is garbage. The 5× speedup isn't theoretical. It sits behind one configuration change most guides don't mention. Those guides target NVIDIA CUDA or AMD ROCm, where the default backend is already the fast one. On Arc, the default is the slow path, and the fast path takes fifteen minutes to set up once.

The Benchmark Gap: IPEX-LLM vs Vulkan on Arc

markaicode tested Arc A770 16 GB at ~70 tok/s via IPEX-LLM Docker (XPU backend) vs 13–15 tok/s via Vulkan on Mistral 7B Q4_K_M, a 5.3× throughput delta. Both numbers reflect sustained generation on a real chat workload with 4K context, not cherry-picked peaks. The A770's 32 Xe-cores and 560 GB/s memory bandwidth give it headroom the B580 doesn't have. The B580's 20 Xe-cores at 2.8 GHz and 456 GB/s still project to 55–65 tok/s via IPEX-LLM. The delta comes from 19% fewer Xe-cores and 18% less bandwidth. At $250 new, the B580 hits $20.8 per GB of VRAM versus the A770's $20.6 at $330. It's the sharper value play if you can accept the 12 GB ceiling. The table below breaks down what each backend delivers: The IPEX-LLM row is what you get after this guide. The 5× gap comes from Intel's purpose-built inference stack. Vulkan is a graphics API that predates matrix acceleration. For Budget Builders, this is the break-even math: fifteen minutes of Docker setup versus months of 77 ms waits per token. If you're cross-shopping, our B580 vs RTX 3060 comparison shows how CUDA's "just works" default compares to Arc's two-path reality, and the hardware pillar explains why we still recommend the B580 despite the setup friction.

Linux Docker Setup — The Primary Path

Docker with the Intel GPU plugin and oneAPI base toolkit are the only host-level dependencies; the rest runs containerized, making this path reproducible across Ubuntu 22.04/24.04, Fedora 40+, and Arch. Start by verifying your kernel has i915 and xe drivers loaded; lspci -k | grep -A 3 "VGA compatible controller: Intel" should show Kernel driver in use: i915 or xe. Install the Intel GPU plugin for Docker, which handles /dev/dri passthrough without manual cgroup rules:

# Ubuntu 24.04 / 22.04 — Intel GPU plugin
sudo apt-get install -y intel-gpu-plugin

# Or via Intel's repository for latest
wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | sudo gpg --dearmor -o /usr/share/keyrings/intel-graphics.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu noble client" | sudo tee /etc/apt/sources.list.d/intel-gpu.list
sudo apt update && sudo apt install -y intel-gpu-tools level-zero

The level-zero loader must be 1.14+. Older versions segfault on XMX initialization. The failure looks like a successful container start with silent GPU fallback to CPU. Verify with dpkg -l | grep level-zero. Next, pull the IPEX-LLM inference image with Ollama preconfigured:

docker pull intelanalytics/ipex-llm-inference-cpp-xpu:latest

Run with device passthrough and the render group permission that --group-add render provides. Omitting this is the #1 cause of "works on host, fails in container" reports:

docker run -it \
 --device /dev/dri:/dev/dri \
 --group-add $(getent group render | cut -d: -f3) \
 -v $(pwd)/models:/models \
 -p 11434:11434 \
 intelanalytics/ipex-llm-inference-cpp-xpu:latest

The container exposes Ollama on port 11434 with the XPU backend already selected. Load Mistral 7B Q4_K_M and verify speed with ollama run mistral:7b-q4_K_M. You should see 55–65 tok/s on the B580 within seconds of the model loading. The first cold-start takes ~8s to load model weights into VRAM. Subsequent loads are near-instant from cache. This setup is reproducible because the image pins the oneAPI runtime, IPEX-LLM libraries, and Ollama build. Your host stays clean, and you can rebuild in 10 minutes on a fresh install.

Windows WSL2 Setup — When You Can't Use Native Linux

WSL2 with Intel GPU driver extension enables Arc passthrough to Linux containers, but performance lags native Linux by 8–15% due to WSL2's GPU virtualization layer and memory copy overhead. Start with Windows driver version 31.0.101.5448 or newer. Arc Control Center will report this in Settings → System → About. The WSL2 kernel as of May 2026 ships at 5.15.146.2, which includes the dxgkrnl paravirtualization driver for GPU access. Update with wsl --update before proceeding.

Inside WSL2, install the same Intel stack but expect tighter version coupling. The level-zero loader in WSL2 must match the host Windows driver's oneAPI version within two minor releases; skew beyond that causes segfaults in XPU initialization that clinfo won't catch because the crash happens in IPEX-LLM's runtime, not the loader itself. Install via:

# Inside WSL2 Ubuntu 24.04
wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | sudo gpg --dearmor -o /usr/share/keyrings/intel-graphics.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu noble client" | sudo tee /etc/apt/sources.list.d/intel-gpu.list
sudo apt update && sudo apt install -y intel-level-zero-gpu level-zero

The Docker run command is identical to native Linux, but add --memory=8g or higher. WSL2's default memory limit chokes IPEX-LLM's allocation. Expect 50–58 tok/s on Mistral 7B Q4_K_M. Cold-start stretches to 12–18s as WSL2 copies weights through the virtualization boundary. The 8–15% penalty is acceptable if your workflow is Windows-native and dual-booting isn't practical. For a dedicated inference box, native Linux recovers that performance for free. Our production Ollama Docker guide covers restart policies and model volume management that apply to both paths.

When Vulkan Is Actually Good Enough

Vulkan setup is zero-overhead: --gpu VULKAN flag in llama.cpp or Ollama's default GPU detection path, no Docker, no driver stack, no group permissions, and functional in under 60 seconds on any Arc driver version. This is the path for quick tests, one-off model evaluations, or machines where you can't justify the IPEX-LLM setup time. The trade-off is explicit: 11–14 tok/s versus 55–65 tok/s, but "explicit" means you chose it knowing the cost. Vulkan is the right tool for verifying whether a model loads at all, or for running inference on a secondary machine where 77 ms/token is acceptable.

The decision tree is simple. For daily inference on your primary rig, use IPEX-LLM Docker. For occasional use, testing, or a machine you don't want to maintain, use Vulkan. At 13 tok/s, the B580's value still holds: $250 for 12 GB VRAM beats renting A100 time on Vast.ai at $2/hour. 125 hours of cloud inference buys this card outright. Vulkan gets you there with zero setup. If you bought Arc for the headline performance numbers, Vulkan is a trap that makes the hardware feel broken. Use it with eyes open, not by accident.

What Fits in 12 GB — Model Size Limits and Offload Rules

Arc B580 12 GB fits Mistral 7B Q8_0 (8.0 GB weights + 2.5 GB KV-cache headroom at 4K context) fully GPU-resident at ~55–65 tok/s via IPEX-LLM; 13B Q4_K_M (7.4 GB weights + cache) also fits without offload. The 12 GB ceiling is the B580's defining constraint versus the A770's 16 GB. Understanding it prevents the performance collapse that happens when you accidentally trigger CPU offload. At 8K context, KV-cache grows to 3.5–4.5 GB, pushing Mistral 7B Q8_0 into the danger zone where a single long conversation suddenly halves your speed. The table below maps practical model choices:

The Q8_0 row is where the B580's value shines: near-uncompressed quality at full GPU speed, something 8 GB cards can't attempt. CodeLlama 13B Q4_K_M is the largest model that fits cleanly. It's useful for coding tasks where 13B parameters outperform 7B on reasoning. The Q5_K_M collapse to 8–12 tok/s happens because PCIe 4.0 x8 bandwidth saturates under 2–4 layers of CPU offload. Weights stream too slowly to keep the GPU fed. The CPU threads stall waiting. For models beyond 12 GB, the hardware pillar covers the RTX 3090 24 GB at $450–500 used, the next step up with enough VRAM to skip these trade-offs. The B580 owns the 7B–13B space at unbeatable $/GB-VRAM.