kingy tested
Tested repair · llama.cpp · b10453 commit3cb7ffb1a1f612d5e4a46244ae5a3c77ad934a70
Check this fix
- Applies when
- cudaMalloc cannot allocate a 1,200 MiB KV-cache buffer.
- Change to inspect
- Change both KV caches to Q8_0 for the same 64K workload, or reduce context to the retained passing 32K FP16 configuration.
- Expected result
- The same 64K workload passed with Q8_0 KV; 32K FP16 KV also passed.
- If it still fails
- Keep the original failed output and check the conditions and verification steps below. Retained failed attempt: 64K FP16 KV allocation failed in the retained run.
Tested or reviewed 2026-08-23. Verify the original task on your own version and settings; this case does not prove a universal fix.
Read the verification steps ↓ · Report your outcome ↓What happened
Symptom
cudaMalloc cannot allocate a 1,200 MiB KV-cache buffer.
Exact environment and tested conditions
Gemma 4 31B-it Q4_K_M
Ryzen 9 7950X / RTX 4090 24 GB / 128 GB RAM
65,536-token allocated context
Correction
Change both KV caches to Q8_0 for the same 64K workload, or reduce context to the retained passing 32K FP16 configuration.
Result
The same 64K workload passed with Q8_0 KV; 32K FP16 KV also passed.
What did not work
64K FP16 KV allocation failed in the retained run.
- Inspect the retained result and receipt before choosing runtime options.
- Keep model, runtime, hardware and other settings fixed when testing the change.
- Select Q8_0 KV cache for the 64K context.
- Record allocation success and task quality separately; retain the failed attempt.
Sources & limits
Gemma 4 31B-it owned RTX 4090 measured result
Record checked 2026-08-23.
Allocation success is not a task-quality assessment.
The exact reviewed hardware configuration matters; available memory differs across systems.