Computers
Peripherals
Components
Supplies
Audio Visual
Mobile Tech
Software
Networking
My products
Product guidesLOCAL AI
The best GPUs for local AI right now
What actually matters when buying a card for running models at home, and an honest look at a market that has rarely been worse for buying one.
There is no comfortable way to open this article. Buying a graphics card in August 2026 is a miserable experience, and anyone telling you otherwise is selling something. The AI-driven memory shortage that started biting in late 2025 has not gone away, NVIDIA has released no new consumer cards this year, and prices across the board sit well above their nominal launch figures.
That does not make the question uninteresting — it makes it more important to know exactly what you are paying for. This article covers what actually matters in a card for local inference, what each of the current options is genuinely good at, and which of the popular "bargains" are traps.
Prices quoted were checked on 28 August 2026 and will not stay accurate for long. Treat them as a snapshot of a moving market, not a price list.
The state of the market, briefly
Tom's Hardware's price tracker, updated on 27 August 2026, has the RTX 5090 changing hands at around $4,829 against a $1,999 launch price — roughly 142% above MSRP. The 5080 sits about 57% over, the 5070 Ti about 49%, the 5060 Ti 16GB about 78%. UK retail tells the same story: on the day of writing, Scan's cheapest RTX 5090 was £3,939.98 and Overclockers' cheapest 5080 was £1,349.99.
The cause is memory. GDDR7 and DRAM supply has been consumed by AI datacentre demand, NVIDIA has reportedly been allocating what it has by revenue per gigabyte, and gaming cards are not where the revenue is. Reporting in February 2026 indicated no new NVIDIA consumer GPUs at all this year, an RTX 50 SUPER refresh designed but deprioritised, and the RTX 60 series pushed towards 2028. AMD has confirmed price increases of at least 10% for the same reason. DRAM contract prices were still climbing 13 to 18% quarter-on-quarter through Q3.
The uncomfortable implication is that waiting is not obviously the winning move. There is no imminent generation to wait for, and the current one is not getting cheaper quickly.
What actually matters
Two numbers, in this order.
VRAM capacity decides what you can run at all. A model that does not fit either spills into system RAM and crawls, or refuses to load. This is a hard wall, not a gradient.
Memory bandwidth decides how fast it runs once it fits. Token generation is memory-bound: the GPU reads through the model's weights for every token produced and spends most of its time waiting on memory rather than computing. Phoronix's llama.cpp benchmarks put the RTX 5090 about 1.58 times ahead of the 4090 on text generation, against a bandwidth ratio of 1.78 — while prompt processing, which genuinely is compute-bound, improved by only around 17%.
Almost everything else — CUDA core counts, ray tracing performance, the marketing TOPS figure — is close to irrelevant for inference. How much VRAM do you actually need works through the arithmetic in detail.
NVIDIA, current generation
Still the path of least resistance. CUDA is the best-supported backend in every runtime, and everything works on day one.
RTX 5090 — 32 GB, 1,792 GB/s, 575 W. The only consumer card that is both large and genuinely fast, and priced accordingly. If money is genuinely not the constraint, this is the answer. For everyone else the price is difficult to justify against two used 24 GB cards.
RTX 5080 — 16 GB, 960 GB/s, 360 W. Excellent bandwidth, but 16 GB caps you below the 27B-and-up class that most of 2026's interesting models target. Superb for 12B-class work, frustrating a year from now.
RTX 5070 Ti — 16 GB, 300 W. The best bandwidth-per-pound in the current NVIDIA line-up, with the same 16 GB ceiling. Currently around £876 in the UK.
RTX 5070 — 12 GB, 250 W. The 12 GB is the binding constraint. Fine for 8B to 12B models, and that is where it stays.
RTX 5060 Ti 16GB — 448 GB/s, 180 W. Interesting for one specific reason: it is the cheapest new way to get 16 GB, at a modest 180 W. The bandwidth is less than half the 5080's, so it will hold the same models and generate noticeably more slowly. For a low-power always-on box that is a perfectly reasonable trade.
Previous generation and the used market
This is where most sensible money goes.
The RTX 3090 remains the value champion for local AI and has for two years: 24 GB of GDDR6X on a 384-bit bus at around 936 GB/s. Ampere retains full CUDA support, it uses ordinary PCIe power connectors, it has display outputs, and it fits in a normal case. XDA put the used price at roughly $600 to $800 in March 2026, and made the obvious point that you can buy two of them for less than one used 5090.
The RTX 4090 is the same 24 GB with more bandwidth — 1,008 GB/s — and correspondingly more money. Also fully supported, also sensible.
A caution on UK used pricing specifically: the aggregators are unreliable right now. One tracker we checked listed the 4090 as more expensive used than new, which tells you what the automated scraping is worth. Check completed eBay listings yourself rather than trusting a price-history graph.
AMD: better than its reputation, with caveats
The hardware is competitive. The Radeon AI PRO R9700 offers 32 GB with ECC at 640 GB/s and 300 W, at a $1,299 MSRP — which, in a market where a 32 GB 5090 is nearly four thousand pounds, deserves a serious look. The RX 7900 XTX has 24 GB at 960 GB/s. The RX 9070 XT is 16 GB at 640 GB/s and was £699.98 in the UK on the day of writing.
The software story is more nuanced than either the enthusiasts or the sceptics tell it. ROCm 7.14 officially supports the RDNA3 and RDNA4 consumer cards, which is a real improvement — but only on Ubuntu 22.04/24.04 and RHEL 9/10. If your homelab runs Debian, Arch, Fedora or Proxmox as the host, you are outside the supported matrix.
More surprisingly, for llama.cpp specifically ROCm is frequently not the fastest option on AMD hardware. Phoronix found the open-source RADV Vulkan driver comfortably beating ROCm 7.1 on text generation on the R9700, and beating the HIP backend on Strix Halo for both generation and prompt processing. A follow-up in May 2026 found ROCm 7.2.3 delivered no llama.cpp improvement over 7.0, with some regressions. ROCm matters much more for PyTorch and training than for GGUF inference.
So: AMD is a genuine option for local inference, the price-per-gigabyte is often better, and the practical advice is to try Vulkan first rather than assuming ROCm is the fast path.
Intel Arc Pro: the interesting outsider
Easy to overlook and worth not overlooking. The Arc Pro B60 is 24 GB at 456 GB/s in a 200 W, PCIe 5.0 x8 card, spotted at around $599 — comfortably the cheapest new route to 24 GB. The Arc Pro B50 is 16 GB with ECC at 224 GB/s in a 70 W package, which is genuinely remarkable for anyone building a low-power always-on inference box. Launch pricing was reported as $299 by Tom's Hardware and $349 by TechPowerUp, so treat that figure loosely.
llama.cpp ships an official SYCL backend for Intel GPUs, with OpenVINO support in progress and Vulkan as a vendor-neutral fallback, and Intel maintains its own ipex-llm project with a llama.cpp quickstart.
The caveat is strategic rather than technical. Intel has reportedly cancelled discrete gaming GPUs for the Xe3P generation, making Battlemage its last. Arc Pro workstation cards and datacentre parts explicitly continue — but you are buying into a line whose long-term consumer future is uncertain.
The datacentre card trap
A used Tesla P40 with 24 GB for a few hundred pounds looks like the obvious homelab hack. It is not, and this is the single most important warning in this article.
NVIDIA has removed Maxwell, Pascal and Volta from the CUDA Toolkit. Drivers still work and already-compiled binaries still run, but future toolkit releases cannot generate code for these GPUs and the accelerated libraries have dropped them. In practice that means a P40 or V100 is on a shrinking software island — new llama.cpp and PyTorch builds will progressively stop supporting it. Turing, which includes the popular T4, has been identified as next in line.
The physical problems compound it. These cards ship with passive heatsinks designed for rack airflow and overheat badly in ATX cases — owners report T4s at 91°C and P40s past 100°C without custom shrouds and high-static-pressure fans. They use 8-pin EPS connectors rather than PCIe, so you need adapters. They have no display outputs. What starts as a $300 card routinely becomes $400 to $500 once cooling and adapters are accounted for.
If you want cheap 24 GB, buy a used 3090.
When a GPU is the wrong answer
If your goal is to run models that will not fit in 32 GB at all, no consumer graphics card solves it and you are into unified-memory territory:
Mac Studio with M5 Ultra — up to 512 GB at 1,200 GB/s. Currently the only machine that is both very large and fast, which is exactly why Apple is marketing it at local LLM users. It is also £5,499 before you add memory.
Mac Studio with M5 Max — up to 128 GB at 614 GB/s, from £2,499.
NVIDIA DGX Spark — 128 GB at 273 GB/s. Worth reading the coverage before buying: John Carmack publicly reported it drawing barely 100 W of its rated 240 W, delivering around half the quoted performance, running hot and rebooting under sustained load. Tom's Hardware's own review was considerably more positive but still noted its bandwidth is far below a Mac Studio's, and that the Mac may well produce more tokens per second despite the Spark's greater raw GPU power.
AMD Ryzen AI Max+ 395 mini PCs — up to 128 GB at 256 GB/s, and generally cheaper than either of the above.
Note the pattern: those 128 GB machines run at roughly a seventh of a 5090's bandwidth. They will hold models a 5090 cannot, and they will do it slowly. That is the trade, and it is a real one — but go in knowing which half you are buying.
External GPUs actually work
A pleasant surprise, and useful for anyone with a small-form-factor server. For inference specifically, the link bandwidth barely matters. XDA tested an RTX 4080 in a Thunderbolt 5 enclosure attached to a handheld running gpt-oss-20b and got over 70 tokens per second — because once the model is in VRAM, the link only carries token data, not gigabytes of weights. Compare that with gaming over Thunderbolt, which costs 5 to 15%.
The link only bites in two places: the initial load of the model into VRAM, and any scenario where the model does not fit and is spilling across the link every token. If it fits, the interface is close to irrelevant. Thunderbolt 5 offers 80 Gbps and hot-plugging; OCuLink offers a direct PCIe 4.0 x4 connection at 64 Gbps with lower overhead but requires a shutdown to connect.
Power and PSUs
Worth checking before you order. NVIDIA's own minimum system power figures are 1000 W for the 5090, 850 W for the 5080, 750 W for the 5070 Ti and 650 W for the 5070 — and those assume a high-end CPU alongside. AMD asks for 750 W for the R9700 and the RX 9070 XT, and 450 W for the RX 9060 XT. NVIDIA specifically recommends a PCIe CEM 5.1 compliant supply for the 5090.
For UK readers there is a further consideration: a 5090 at 575 W in a 1000 W-minimum system is a meaningful fraction of what a single 13 A socket will deliver. A multi-GPU rig needs the circuit thought about deliberately, not discovered later.
What we would actually buy
Best value overall: a used RTX 3090. 24 GB, fully supported, ordinary power and cooling. Two of them still cost less than one 5090.
Best new card under £900: RTX 5070 Ti, accepting the 16 GB ceiling — or an Arc Pro B60 if you want 24 GB more than you want a frictionless software experience.
Best low-power always-on: Arc Pro B50 at 70 W, or an RTX 5060 Ti 16GB at 180 W if CUDA compatibility matters to you.
Best value per gigabyte, new: Radeon AI PRO R9700 — 32 GB with ECC — provided your host OS is on ROCm's supported list, and provided you are willing to test the Vulkan backend.
If you need more than 32 GB: a unified-memory machine, with clear expectations about bandwidth.
What not to buy: a used Tesla P40 or V100. The CUDA support is going away.
Once you have hardware, Running local AI in a homelab covers passing it through to a VM and making it a proper service rather than a desktop toy. If you have not installed anything yet, the Ollama guide is the place to start.