Local AI memory guide
There is no universal RAM number for local AI. Start with the exact model artifact, then account for context, runtime allocations, the operating system, and whatever else must stay open.
The practical answer
For the LM Studio desktop app, the vendor recommends at least 16GB of RAM on supported Windows systems and 16GB or more on Apple Silicon Macs. Its documentation says an 8GB Mac may still run smaller models with modest context sizes. Those are application recommendations, not proof that every model will fit.
For a real compatibility decision, calculate from the model you will load. The weights must fit somewhere, and the runtime also needs memory for context or KV cache, working buffers, optional vision components, and other allocations. The operating system and normal applications need their own headroom.
Do not buy from a parameter count alone. Quantization, model architecture, context length, runtime, and offload strategy can make two “8B” deployments use different amounts of memory.
What consumes system or unified memory
Weights
The selected GGUF, MLX, ONNX, or other model artifact is the largest predictable starting allocation. Its download size is useful evidence, but not the total runtime footprint.
Context cache
The KV cache lets the model reuse attention state across a prompt and response. Longer context generally requires more memory, with the exact amount determined by the model and cache settings.
Runtime buffers
Inference engines allocate memory beyond the weights. Batch size, parallel requests, flash attention, multimodal projectors, and backend choices can change that overhead.
CPU-offloaded layers
When a discrete GPU cannot hold the full model, some runtimes can keep part of it in system RAM. That can make a model runnable without making it fast.
macOS unified memory
On Apple Silicon, the CPU and GPU access one shared pool. Model memory competes with macOS and every other process rather than occupying a separate dedicated VRAM pool.
Application headroom
Browsers, development tools, document indexes, and other workloads can be substantial. A configuration that only fits after closing everything may be a poor everyday setup.
Why model format changes the answer
The llama.cpp quantization documentation gives a concrete illustration using Llama 3.1: its listed 8B Q4_K_M artifact is 4.9GB, while the listed 70B Q4_K_M artifact is 43.1GB. These figures describe the example weight files, not the total RAM required for an arbitrary context or runtime.
Current model registries show the same basic pattern across variants. For example, Ollama lists Qwen3 artifacts ranging from hundreds of megabytes to more than 100GB. The useful number is the exact tag and file you intend to run, not the family name by itself.
| Evidence | What it tells you | What it does not tell you |
|---|---|---|
| Model artifact size | The storage occupied by the selected weights and related files. | Total runtime memory, context-cache size, or speed. |
| Runtime estimate | A configuration-specific projection including settings the estimator understands. | How other apps, future requests, or different settings will affect the machine. |
| Successful load | The model fits with that runtime and configuration. | Whether it is fully GPU-accelerated or responsive enough for the intended task. |
| Measured memory during work | The best evidence for the exact model, context, concurrency, and workload tested. | A guarantee for another model version, runtime release, or device. |
Context length can move the limit
A model may advertise a very large maximum context window, but enabling that maximum is a separate memory decision. Ollama explicitly warns that increasing context length increases memory use. LM Studio’s resource estimator accepts the intended context length, and its estimator also accounts for settings such as GPU offload, flash attention, and vision support.
Start with the smallest context that supports the task, verify the actual allocation, and increase only when the workload needs it. Long-document analysis and coding agents can need far more context headroom than a short private chat.
A defensible RAM-planning process
- Choose the exact runtime, model tag, artifact, quantization, and modality.
- Record the model file size and the context length you genuinely need.
- Use LM Studio’s estimate-only command or the equivalent estimator in your runtime before loading.
- Leave headroom for the operating system and the applications that will remain open.
- Load the model, run a representative prompt, and inspect actual placement and memory pressure.
- Repeat the check after changing context length, GPU offload, cache precision, parallelism, or model version.
Buying guidance without a fake universal tier
If local AI is a primary use, avoid a configuration that only meets the model’s weight size on paper. Favor enough memory for the intended model and context plus normal work, and remember that soldered or non-upgradeable memory turns an estimation mistake into a permanent constraint.
For a Windows laptop with a discrete GPU, evaluate RAM and VRAM separately because the model may be split between them. For Apple Silicon, evaluate the single unified-memory pool as a shared budget. In both cases, an exact runtime estimate is more useful than a generic “small, medium, or large model” label.
RAM for local AI FAQ
Is 16GB enough for local AI?
It meets LM Studio’s stated recommendation for its supported Mac and Windows configurations, but it does not guarantee that a particular model, context, or workload will fit. Check the exact artifact and estimate.
Can virtual memory or swap make a larger model usable?
It may prevent an immediate allocation failure in some configurations, but heavy swapping can make interactive inference impractical. Treat measured responsiveness—not a successful launch—as the useful result.
Should RAM equal the model download size?
No. The model file is only the starting allocation. Context cache, buffers, runtime behavior, and the rest of the system require additional memory.