Quick overview
System RAM vs GPU VRAM for local LLMs: when CPU offload helps, when it does not, and how to size both on one PC.
GPU VRAM holds the layers your runtime keeps on the card. System RAM holds the OS, apps, and any offloaded model layers the engine parks on the CPU. Confusing the two leads to expensive RAM upgrades that never fix an OOM error on the GPU.
Quick answer
Fit as many layers as possible in VRAM for speed. Use system RAM only for spillover you accept will be slower. If the full model does not fit across VRAM plus RAM, shrink the model or quant.
Speed hierarchy
1. Full GPU fit (fastest interactive use) 2. Partial offload to RAM (mixed) 3. CPU-only (slowest for large models)
Sizing system RAM
- 16 GB: tight for desktop plus inference
- 32 GB: comfortable default for experimentation
- 64 GB+: heavy offload, VMs, or multi-user lab machines
Upgrade decision tree
- OOM on GPU with empty-looking task manager GPU tab: need more VRAM or smaller model
- OOM mentioning CPU RAM during offload: add system RAM or reduce offload layers
Related guides
VRAM sizing and RAM compatibility checks.
How Roger N Reckon can help
Roger N Reckon installs RAM upgrades and can advise whether your bottleneck is VRAM or system memory after a short diagnostic. See upgrade services for routing.
Practical checklist
1. Write down the model size, quant level, and target context length you need. 2. Measure current GPU VRAM, system RAM, and free NVMe space before buying parts. 3. Confirm PSU wattage against the GPU vendor's required system power. 4. Plan cooling for sustained inference, not just short benchmarks. 5. Back up data before OS or hardware changes. 6. Request a written component list, testing scope, and warranty terms for any build or upgrade. 7. Re-test with the same prompt and context after changes.
Frequently Asked Questions
Can 128 GB RAM replace a GPU?
No for interactive GPU-accelerated inference. It only helps offload scenarios.
Does faster RAM speed tokens?
Helps CPU offload somewhat; VRAM bandwidth matters more for GPU layers.
Is DDR5 required?
Follow motherboard platform; DDR5 is common on new builds.
How much RAM for 70B offload?
Often impractical on consumer desktops; plan data-center memory or cloud.
Will closing Chrome free VRAM?
Browser uses GPU memory on some systems; close GPU-heavy tabs before inference.
Final takeaway
VRAM is speed; RAM is spillover and multitasking. Upgrade the right pool.