The point
- VRAM capacity decides whether the model + context stay on the GPU. Spill to system RAM and decode can fall from ~12–15 tok/s to under 1 tok/s.
- Memory bandwidth decides tok/s once it fits. Extra VRAM does not rewrite a slow memory bus.
- B70-class (~32GB, ~608 GB/s) owns the speed lane for anything that fits in 32GB (~55–70 tok/s on tuned ~35B MoE Q4).
- Spark / Halo / Mac (~128GB unified) own capacity for dense 70B+ — but Spark (~273 GB/s) often lands only ~3–8 tok/s on dense 70B batch-1, while Mac Max (~546–614 GB/s) is more like ~10–15 tok/s.
- There is still no clean “UMA desktop you stick a B70 into.” Hybrid offload is capacity rescue, not full-GPU speed.
Rule of thumb: B70 for ≤32GB speed; Spark for NVIDIA big-model desk work; Halo for value 128GB x86; Mac Max for fastest chat decode in the unified-memory club. Want both speed and 70B+ fit? Two machines (or two cards), not one mythical hybrid.
One chart
| RTX 5080 (house) | Arc Pro B70 | DGX Spark | Strix Halo | Mac Max | |
|---|---|---|---|---|---|
| Memory | 16GB GDDR | 32GB GDDR | ~128GB unified | ~128GB UMA | ~128GB unified |
| Bandwidth | high GDDR (card-class) | ~608 GB/s | ~273 GB/s | ~256 GB/s | ~546–614 GB/s |
| 27B IQ4 short ctx | ~12–15 tok/s when fit; <1 after spill | fits + fast | easy capacity | easy capacity | easy + fast |
| 27B IQ4 long ctx | ~32–64k clean; 128k spills (~18GB reported) | ~128–256k on-GPU (same weights) | not the limiting factor | not the limiting factor | not the limiting factor |
| Dense ~70B tok/s | N/A alone | N/A alone | ~3–8 (some FP8 ~2.7) | ~6–12 | ~10–15 |
| ~35B MoE Q4 tok/s | FreeToken MoE smoke: wall ~60–90 on short prompts when it worked | ~55–70 tuned | tens with CUDA/TensorRT | mid-teens to low-tens | often ahead of Spark |
| Best at | Daily coding on 16GB if you stay honest about ctx | Max tok/s inside 32GB | Big models + CUDA tooling | Value 128GB x86 | Chat decode champ (UMA class) |
5080 rows are house Ollama / FreeToken readings. Spark / B70 / Halo / Mac tok/s are community ballparks from last30days (2026-09-09). Stack and quant move the numbers.
What the 16GB → 32GB question really answers
For a ~27B IQ4_XS model (weights ~13–14GB) with q8_0 KV: Ollama SIZE scaled about 14GB at 32k, 18GB at 128k, 24GB at 256k. Doubling to 32GB mostly removes the spill. You get ~128–256k on-GPU. You do not get a free tok/s rewrite — long prompts still cost first-token time — but you keep native decode (~10–15 tok/s on this class of card) instead of falling off the cliff.
Spend that 32GB on denser weights (Q5/Q6/Q8) and you buy less context. BF16/FP16 27B still will not fit (~50GB+ weights).
Why “UMA + stick a B70 in it” fails
Sealed UMA boxes (Spark GB10 desk, Framework / Halo minis, Mac) do not take a fat dGPU. Real towers take a B70-class card but use classic DDR + VRAM, not Mac-style UMA. Near-misses like a mini with a weak PCIe link and small PSU are not a 230W GPU home.
Ollama hybrid offload (layers on GPU + UMA/DDR) is how oversized models run. Speed falls toward the slow path — often roughly one-third to one-half of pure-GPU tok/s, sometimes worse. Capacity rescue, not B70 feel.
What to buy for which job
- Chatty models ≤32GB: tower + B70-class (or keep the 16GB 5080 and cap context).
- 70B+ without multi-GPU: Spark if you need CUDA; Halo for value x86 UMA; Mac Max if decode speed wins.
- Both: two boxes (fast ≤32GB daily driver + 128GB overflow), or two cards — not one hybrid SKU.
Related: research swarm, Gemma vs IQ4, quant bar, FreeToken MoE.
Sister site: toklanes.com — measured tok/s review-bench.