Entry
8–16 GB RAM · integrated graphics
1B–3B quantized
Good for private drafting, summaries, and lightweight coding helpers. Expect CPU-speed responses and keep context modest.
Buying guide
Buy memory first. Then everything else.
Local inference is mostly a memory-capacity and bandwidth problem. These tiers map the spec sheet to workloads you can realistically enjoy.
Mid range
16–32 GB RAM · 8 GB VRAM
7B–13B quantized
The local-LLM sweet spot. A modern 8 GB GPU runs capable 7B/8B models comfortably and some 14B models tightly.
High end
32–64 GB RAM · 16–24 GB VRAM
Up to 32B comfortably
Fast enough for stronger reasoning and coding models. Some 70B Q4 workloads can spill to system RAM, but speed falls.
Workstation
64–192 GB RAM · multi-GPU
70B+ and higher precision
For large dense models, long contexts, batching, and throughput. Memory bandwidth and interconnect now matter as much as capacity.
Apple Silicon
One pool. A real advantage.
Apple’s CPU and GPU share unified memory. A 64 GB Mac can make most of that pool available to model weights, where a PC’s 64 GB system RAM and 8 GB GPU are separate. It does not make the model smaller, but it can avoid slow transfers and let larger models use the GPU.
Discrete PCSeparate pools
8 GBVRAM
32 GBSystem RAM
Apple SiliconUnified pool
32 GB sharedCPU + GPU access
The vocabulary
Five things worth knowing.
01
Parameters
The learned weights inside a model. More parameters usually mean more memory, not automatically more useful output.
02
Quantization
Storing weights at lower precision. Q4 uses roughly a quarter of FP16 weight memory, with a modest quality tradeoff.
03
Context window
How much text the model can keep in working memory. Longer context grows the KV cache and adds memory beyond the weights.
04
CPU vs GPU
CPU inference is broadly compatible but slower. A GPU provides much higher memory bandwidth and parallel compute.
05
Tokens per second
The practical speed measure. Capacity tells you if a model loads; bandwidth and compute tell you how fast it responds.