Toolchain-first selection
Developer hardware is compatible only when the exact framework, runtime backend, driver, compiler, container path, profiler, and deployment target work together.
Freeze the stack you must support
Record host architecture, operating system and build, kernel where relevant, firmware, driver, toolkit, framework and version, runtime, execution provider, compiler, Python or other language version, package source, model format, container engine, editor or notebook, profiling tools, and deployment target. Decide which versions must match production and which may differ deliberately.
A detected GPU is not an end-to-end compatibility test. The framework build, operator set, device architecture, driver, user-space libraries, container runtime, and application must all select the intended device. Keep CPU fallback visible.
Verify the backend at operator level
Framework build
Use the framework’s current installation selector or support matrix for the exact operating system, package, language, and compute platform. Do not infer support from the accelerator family alone.
Execution provider
ONNX Runtime partitions supported nodes or subgraphs among registered providers in priority order. Inspect the active provider and any nodes left on the CPU.
Driver and toolkit
Match the device, driver, operating system, kernel, toolkit, framework, and container requirements in the current vendor documentation. Preserve the tested combination.
Target parity
Decide whether development must match the deployment device, instruction set, memory architecture, precision, provider, container, and performance constraints—or whether cross-target tests cover the difference.
Containers do not virtualize away the accelerator stack
Docker Compose can reserve GPU devices only when the host contains them and the daemon is configured with the required support. NVIDIA’s Container Toolkit documentation likewise requires a supported platform and host driver before the runtime exposes devices to a container. Treat the host driver, runtime integration, image libraries, device reservation, permissions, and container test as separate gates.
Pin image digests or immutable releases where reproducibility matters, document the build context and package indexes, generate a software bill of materials where required, and retain a clean rebuild test. A notebook that works after months of unrecorded interactive changes is not a reproducible environment.
Profile placement and peaks
Record model weights, optimizer state and gradients when training, activations or workspace, input and output tensors, compilation caches, context or KV cache where relevant, data-loader buffers, host-pinned memory, operating-system use, and simultaneous tools. Dedicated device memory and host RAM remain distinct unless the documented platform and framework expose another architecture.
PyTorch Profiler can collect CPU and device activity, operator shapes, stack traces, timing, and memory information. Use equivalent tools for the chosen stack to locate transfers, fallback, synchronization, compilation, and the actual peak. Start with the Local AI Compatibility Guide when you need the broader device-verification method.
Developer candidate worksheet
| Area | Evidence to collect | Acceptance gate |
|---|---|---|
| Install and build | Clean OS image, driver and toolkit versions, compiler, package manager, lockfiles, container files, architecture, and build log. | A clean machine or image reproduces the environment without undocumented manual repair. |
| Runtime device | Framework build, registered backend or provider, operator coverage, device selection, fallback policy, and profiler trace. | The representative model uses the intended device; unexpected fallback fails the gate. |
| Memory and data | All host and device allocations, dataset or model storage, cache, checkpoint, I/O pattern, and concurrency. | The worst representative run fits with recorded margin and restores from a checkpoint where required. |
| Reproducibility | Code revision, data revision, model artifact hash, environment, seeds, deterministic settings, hardware identity, and expected tolerances. | Repeated tests meet the declared tolerance on the supported hardware/software combination. |
| Profiling | Warm-up method, trace schedule, operator shapes, CPU and device activity, memory peaks, transfers, and power mode. | The bottleneck and device placement are evidenced, not inferred from utilization alone. |
| Deployment parity | Target architecture, runtime, provider, precision, operators, memory, container, latency, concurrency, and update method. | Exported artifacts pass functional and performance tests on the actual deployment target. |
Define reproducibility narrowly and honestly
PyTorch states that complete reproducibility is not guaranteed across releases, commits, platforms, or CPU and GPU executions, and that deterministic operations can be slower. Therefore record the scope: same code, data, artifact, software stack, hardware, device path, and tolerance. Separate exact repeatability from statistically equivalent model quality and from performance repeatability.
Let deployment and service constraints choose the chassis
For operating-system and accelerator-path differences, use Mac vs Windows AI PC for Local AI and GPU vs NPU for AI. If component replacement and expansion matter, use the AI Desktop Buying Guide; if validated configurations, high capacity, remote management, and service response matter, use the AI Workstation Buying Guide.