Local AI compatibility
A practical way to answer “can this computer run local AI?” without pretending that a processor label, parameter count, or memory total settles the question.
The short answer
A machine is compatible only when a specific model, format, runtime, operating system, and execution backend work together with enough memory for the chosen context and workload. “AI PC,” “Apple Silicon,” and “has a GPU” are useful clues, but none is a complete compatibility result.
Use the exact artifact, not the marketing category. A 4-bit model file and an unquantized model with the same parameter count can have radically different memory requirements. Context length, vision components, concurrent requests, and CPU or GPU offloading can change the result again.
A five-step compatibility check
- Name the workload. Short private chat, long-document analysis, coding agents, image generation, and model training stress hardware differently.
- Choose the exact model artifact. Record its family, variant, quantization, file size, modalities, and intended context length. Parameter count alone is not enough.
- Choose the runtime and backend. Confirm that the current runtime supports the operating system and the intended CPU, GPU, Metal, CUDA, Vulkan, DirectML, or NPU path.
- Estimate the whole memory load. Include model weights, runtime allocations, context or KV cache, multimodal components, the operating system, and other applications.
- Verify after loading. Use the runtime’s estimator and status tools to confirm where the model actually loaded. A successful launch does not prove that it is fully accelerated or suitable for the workload.
The memory terms that matter
Model weights
The downloaded artifact is the starting point. Quantization reduces weight storage, but lower precision can change output quality and runtime support.
System RAM
Windows and Linux systems use RAM for the operating system, CPU inference, CPU-offloaded layers, and application overhead.
Dedicated VRAM
A discrete GPU has its own memory pool. Full GPU offload needs room for the selected weights plus the GPU-side runtime and cache allocations.
Unified memory
Apple Silicon exposes one shared pool to the CPU and GPU. It is not a separate VRAM amount that can be added to system memory.
Context and KV cache
Longer prompts and conversations require additional memory. The amount varies by model architecture, precision, runtime, and cache settings.
Headroom
A model that barely fits may cause swapping, partial offload, instability, or poor responsiveness when the operating system and other apps compete for memory.
What current runtimes can verify
| Runtime or framework | Useful evidence | Important limit |
|---|---|---|
| LM Studio | lms load --estimate-only estimates memory for the exact model and honors context-length and GPU-offload settings. |
An estimate is configuration-specific; changing context, vision support, or offload changes the answer. |
| Ollama | ollama ps reports whether a loaded model is on the CPU, GPU, or split between them. |
Loading successfully may still mean partial CPU offload rather than full GPU acceleration. |
| llama.cpp | The project documents supported backends, quantization, and CPU+GPU hybrid inference. | Backend support is not a performance promise for every model or device. |
| Windows ML | Windows can use managed execution providers for supported CPU, GPU, and NPU paths. | The model must be compatible with the chosen runtime and execution provider. |
| MLX LM | Apple’s MLX stack supports generation and quantized models on Apple Silicon. | Unified memory is shared with macOS and other applications. |
Compatibility guides
How much RAM for local AI?
Plan system and unified memory around an exact model, context, runtime, and the applications that must remain open.
How much VRAM for local AI?
Understand full GPU offload, partial offload, context memory, and image-generation tradeoffs.
Can a Mac run local LLMs?
Check Apple Silicon, unified memory, Metal or MLX support, and the actual model footprint.
Can an AI laptop run local LLMs?
Separate CPU, GPU, and NPU capability instead of treating an “AI laptop” badge as universal support.
Can an AI Mini PC Run Local LLMs?
Check the exact model, runtime backend, memory placement, cooling, and upgrade limits of a compact system.
Can a Phone Run Local AI?
Assess mobile runtimes, model formats, shared memory, thermal limits, and battery impact on Android or iOS.
Local AI Setup Guide for Beginners
Start with one task, one supported runtime, and one model artifact, then verify memory and hardware placement.
Local AI Setup Guide for Creators
Plan language, image, transcription, and multimodal workloads around the creative applications that must stay open.
Local AI Setup Guide for Small Businesses
Design a maintainable local service around approved use, access control, capacity, monitoring, and rollback.
How this guide stays maintainable
Compatibility statements here must be traceable to current runtime documentation, an exact model artifact, or a reproducible local observation. Vendor system requirements are labelled as vendor requirements. File size is not presented as total memory use, and a theoretical accelerator specification is not treated as proof that a model runs.
Runtime and hardware support changes. Recheck the linked documentation before buying hardware or standardizing a deployment, especially when a page names a driver, operating-system release, model variant, or preview feature.
Local AI compatibility FAQ
Does an NPU guarantee that a laptop can run any local LLM?
No. An NPU needs a compatible model format, runtime, driver, and execution provider. Some local LLM software may instead use the CPU or GPU.
Is the model download size the same as required RAM or VRAM?
No. The weights are only one allocation. Context cache, runtime buffers, vision components, and the operating system add memory use.
Does “100% GPU” prove that performance will be good?
No. It confirms placement, not speed, output quality, thermals, or suitability for your workload. Measure the task you actually plan to run.