Workload-first selection
The right local AI computer is the smallest defensible system that runs the exact workload, through the intended software path, for the required session and service conditions.
Define the workload before the computer
Record the model artifact or pipeline, runtime and version, operating system, execution backend, precision or quantization, input dimensions, context, batch, concurrency, output requirement, session duration, connected applications, and whether the job must run offline. A product family, accelerator badge, or advertised operations-per-second figure does not establish that this packet will execute on the intended device.
Buy against a repeatable acceptance packet. Keep the model files, configuration, representative inputs, expected outputs, logs, measurement method, and pass criteria. Re-run it on the orderable configuration—not merely on a similar review unit.
Prove the software and device path
Runtime and model format
Confirm that the exact runtime reads the artifact, supports its architecture and operators, and documents the target operating system and processor architecture.
Execution backend
Record the selected CUDA, ROCm, Metal, DirectML, Core ML, OpenVINO, NPU, Vulkan, or CPU path. ONNX Runtime can assign only supported nodes or subgraphs to an execution provider, so partial fallback must remain visible.
Drivers and versions
Treat the device, driver, firmware, operating-system build, runtime, and application as one compatibility set. Recheck the current vendor matrix before ordering or upgrading.
Evidence of placement
Use runtime logs, provider lists, device monitors, and profiling to prove where weights and operations execute. A completed output alone may hide CPU fallback.
Separate every memory pool
Model weights are only one allocation. Add runtime overhead, activations or workspace, context and KV cache where relevant, temporary buffers, input and output media, operating-system use, and every concurrent application. Dedicated VRAM, system RAM, unified memory, storage, and an NPU’s accessible memory are not interchangeable labels.
llama.cpp documents CPU-plus-GPU hybrid inference for models larger than available VRAM. MLX documents direct CPU and GPU access to one unified-memory pool on Apple silicon. Both can be useful, but neither removes the need to measure the complete working set and any performance cost of transfers, offloading, or memory pressure. Use the Local AI Compatibility Guide for the detailed verification method.
Choose the form factor after the packet fits
| Form factor | Defensible when | Inspect before purchase |
|---|---|---|
| Laptop | The workload and its display, input, battery, and travel needs must move together. | Exact configuration, plugged-in and battery behavior, memory ceiling, cooling, ports, repair, and the AI Laptop Buying Guide. |
| Mini PC | A compact stationary node fits the measured workload and service plan. | Runtime support, shared or dedicated memory, sustained cooling, storage, networking, ports, and the AI Mini PC Buying Guide. |
| Desktop | Component choice, cooling, storage, accelerator replacement, or later expansion has measurable value. | Slot and lane wiring, chassis clearance, power, thermals, memory topology, and the AI Desktop Buying Guide. |
| Workstation | Validated configurations, capacity, service response, remote management, or lifecycle controls justify the premium. | Certification scope, support matrix, error controls, multi-device behavior, spares, and the AI Workstation Buying Guide. |
| Phone or edge device | The bounded model, latency, sensor, power, privacy, and offline requirements are proven on the exact device. | Deployable model format, operators, accessible accelerator, thermal behavior, update path, and data lifecycle. |
Size for the actual service
An interactive single-user session, a scheduled batch job, and a shared endpoint impose different requirements even when they use the same model. Record arrival rate, simultaneous requests, queue limit, latency target, throughput target, uptime window, maintenance window, storage growth, and recovery time. Then test the worst representative mix for long enough to expose memory growth, thermal limits, throttling, and error recovery.
If the workload can be split between local and hosted infrastructure, compare the data flow, network dependency, administration, capacity peaks, and exit path with Local AI vs Cloud AI Hardware.
Acceptance gates
Identity gate
The delivered CPU, accelerator, memory, storage, networking, firmware, operating system, driver, runtime, and warranty match the approved bill of materials.
Functional gate
The exact artifacts load, required operators run on the intended device, outputs meet the recorded criteria, and logs expose any fallback.
Capacity gate
Peak memory, context, batch, concurrency, storage, and connected-application use fit with recorded operating margin.
Sustained and recovery gate
The representative session meets latency, throughput, temperature, noise, power-state, error, restart, backup, and restore requirements.