Buy a used RTX 3060 12 GB ($220) or RX 6700 XT 12 GB ($180). Pair it with a $60 used Ryzen 5 3600 and 32 GB DDR4. You can run 13B models at Q4_K_M with 8–15 tok/s. The RTX 3060 wins for CUDA compatibility. The RX 6700 XT saves $40 but needs ROCm patience. Skip the RTX 3060 Ti 8 GB — the 4 GB VRAM gap costs you a full model tier. New 2026 cards (RTX 5060 8 GB, RX 9060 XT 8/16 GB) are tempting but 8 GB still traps you at 7B; wait for the 16 GB RX 9060 XT at $349 if you can stretch.
The GPU: Your Most Important Decision
VRAM-per-dollar is the entire game when you're squeezing local LLM performance out of a $500 build. At $180–220 on the used market as of mid-2026, the RTX 3060 12 GB delivers 17.1 GB/$, double what any new 8 GB card can offer. Compare that to the RTX 5060 8 GB at $299 (8.0 GB/$) or the RX 9060 XT 8 GB at the same price (also 8.0 GB/$). The math is brutal. You pay 50% more money for 33% less VRAM. That 4 GB gap isn't academic. A 13B-parameter model at Q4_K_M quantization needs roughly 8.1 GB. That fits comfortably inside the 3060's 12 GB. You still have headroom for OS overhead. On an 8 GB card, you're hard-capped at 7B models — a full tier below. The only exception is the RX 9060 XT 16 GB at $349. It hits 22.9 GB/$ and matches the 3060's 13B capability. But that price consumes 70% of your total build budget. You still need a CPU, motherboard, and RAM.
If you need CUDA compatibility, pick the RTX 3060 12 GB. llama.cpp works. Every quantization format is supported. You won't spend weekends debugging ROCm drivers. If you're comfortable with Linux and don't mind trading some setup pain for $40 in savings, the RX 6700 XT 12 GB (~$180 used) offers identical VRAM capacity but requires the HSA_OVERRIDE_GFX_VERSION workaround and accepts that some llama.cpp features lag behind CUDA. Do not buy an RTX 3060 Ti 8 GB because "it's faster." The Ti's memory controller cuts VRAM to 8 GB. In local LLM inference, VRAM capacity beats raw TFLOPS every time. A 3060 Ti benchmarks higher in traditional games. But it chokes on the same 7B ceiling as a GTX 1070. The vanilla 3060 12 GB already generates your 13B responses.
Full Build Parts List
Let's assemble the actual PC. The Intel path starts with a used RTX 3060 12 GB ($180–220). Add a new Intel Core i3-12100F ($89), an H610 motherboard ($75), 16 GB DDR4-3200 ($35), a 512 GB NVMe SSD ($38), a 500 W 80+ Bronze PSU ($45), and a budget case ($25). Total: $487–527. The 12100F is a four-core Alder Lake chip with a 58 W TDP. Local LLM inference is GPU-bound after prompt processing. This CPU won't bottleneck generation speed. PSU sizing matters. The RTX 3060 pulls 170 W at full load. The 12100F hits 58 W. Add 20% headroom per OuterVision's calculator. You land at roughly 274 W sustained. That sits well inside a 500 W unit's capacity. Don't cheap out on a no-name PSU; a fire-prone $30 unit that takes your $200 GPU with it isn't savings.
The AMD alternative swaps to a Ryzen 5 5600 ($98) and B450 motherboard ($65). That costs $2 more total. It is the smarter long-term play. PCIe 4.0 support on B450 (via BIOS update) gives you a resale path that H610 lacks. The 5600's six cores handle background tasks more gracefully. You can run a local vector database or browser tabs alongside inference. Either build needs the same GPU, RAM, storage, PSU, and case. After assembly, grab our llama.cpp setup guide — the Windows CUDA path is three commands, or go Linux if you want maximum VRAM efficiency. Verify that your GPU is recognized:
nvidia-smiYou should see your 3060 12 GB listed with ~11.8 GB available. Then test inference with:
llama-cli --model llama-3.1-8b-Q4_K_M.gguf --gpu-layers 99 --prompt "Write a Python function to sort a list"If you see CUDA:0 in the backend initialization and tok/s counts above 25, your build is working.
What This Build Actually Runs
The numbers that matter are generation speed — how fast words appear — and prompt processing speed — how fast the model ingests your context. On our standard bench script (llama.cpp CUDA backend, commit 2026-04), the RTX 3060 12 GB paired with that i3-12100F and 16 GB DDR4-3200 hits 28–32 tok/s generation and 42–48 tok/s prompt processing on Llama 3.1 8B at Q4_K_M, batch=512, context=4096. That's conversational speed; you're not waiting. Step up to 13B Q4_K_M and generation drops to 14–17 tok/s — still usable, still faster than reading aloud. At Q5_K_M quality on 7B, expect 22–25 tok/s. Use our VRAM calculator before downloading models: enter your GPU, pick a model, and it'll flag whether you have headroom or are walking into an out-of-memory crash.
Any 8 GB card — RTX 5060, RX 9060 XT 8 GB, RTX 3060 Ti, doesn't matter — is capped at 7B Q4_K_M (~4.7 GB weights plus ~2.5 GB OS overhead). You cannot run 13B. You cannot run 8B at higher quality quantizations. The RX 9060 XT 16 GB breaks this pattern. It projects 18–22 tok/s on 13B Q4_K_M. But that projection extrapolates from RX 7900 XTX 24 GB benchmarks at 65% proportional core count. No verified llama.cpp ROCm numbers exist as of mid-2026. Buy it for the VRAM, not the speed promise.
| Model + Quant | VRAM Used | Generation Speed | Works on 8 GB? | Works on 12 GB? |
|---|---|---|---|---|
| Llama 3.1 7B Q4_K_M | ~4.7 GB | 28–32 tok/s | Yes | Yes |
| Llama 3.1 8B Q4_K_M | ~4.7 GB | 28–32 tok/s | Barely (no headroom) | Yes |
| Llama 3.1 13B Q4_K_M | ~8.1 GB | 14–17 tok/s | No | Yes |
| Llama 3.1 7B Q5_K_M | ~5.5 GB | 22–25 tok/s | No | Yes |
| Llama 3.1 13B Q5_K_M | ~9.8 GB | 10–13 tok/s (est.) | No | Yes |
New 2026 Cards: Should You Wait?
NVIDIA and AMD both launched $299 cards in 2026, and on paper they're tempting. The RTX 5060 8 GB arrived in May with architectural improvements; the RX 9060 XT 8 GB follows in June. Both lose to a two-year-old used GPU. The 5060's 8 GB at $299 is 8.0 GB/$, half the 3060 12 GB's 17.1 GB/$. Worse, it's still an 8 GB card, still capped at 7B parameters. You're paying a $80–120 premium for a "new" sticker and a smaller power draw that doesn't change what models you can run. For local LLMs, that's performance theater. It looks good in a spec sheet. It changes nothing in your terminal.
The RX 9060 XT 16 GB at $349 is the only 2026 card worth considering, and only in a narrow circumstance. Its 22.9 GB/$ actually beats the used 3060, and it matches the 13B capability. But $349 is 70% of a $500 build before CPU, motherboard, RAM, storage, PSU, and case. That math only works if you already own a compatible AM5 motherboard, a Ryzen 7000 CPU, and DDR5. You are upgrading a GPU, not building fresh. For the true $500 builder starting from zero, the used RTX 3060 12 GB remains the only card that leaves enough budget for a complete, working PC. See our full hardware pillar if you want to compare all tiers from $500 to $3,000.
The Honest Verdict
At $500, buy the used RTX 3060 12 GB today. Don't wait for the RTX 5060 8 GB. Don't wait for the RX 9060 XT 8 GB. The 12 GB VRAM ceiling unlocks 13B-parameter models that 8 GB cards cannot run. The $80–120 you save versus new 2026 cards pays for your CPU, motherboard, RAM, storage, PSU, and case. That isn't compromise. It is the only configuration that hits the budget while delivering usable local inference. The 3060's CUDA compatibility means you'll spend hours running models, not debugging drivers. Its power draw (170 W) is manageable in cheap cases with basic airflow. And the used market has enough volume that you can be picky: buy from sellers with return policies, verify the card's VRAM with nvidia-smi on receipt, and reject anything that smells of mining abuse.
The only reason to deviate is if you already own a modern AMD platform and can stretch to $349 for the RX 9060 XT 16 GB. It matches the 3060's 13B capability with newer efficiency. The extra 4 GB gives you headroom for larger context windows or Q5_K_M quality on 13B. But at full platform cost, it breaks $500. This guide promises a working build at that price. It is not an aspiration that needs another $200 you don't have. Start with the 3060. Run 13B models tonight. Upgrade later when your use case demands it, not because a launch calendar told you to.