AI News

Best Local AI Models for 32GB: RTX 5090-Class Cards and 32GB Macs

Short answer: on a 32GB graphics card, run Qwen3.6 35B A3B. It tied for the top score in Kingy Lab’s 120-task test (98%), scored 89.7 on our board, wrote 62.0 tokens a second on real jobs on our 128GB test machine, and fits with 5.4 GB to spare, even counting the vision file Ollama loads with it. On a 32GB Mac it doesn’t fit by our rule; pick Qwen3.8 27B (the 2-bit file has the top board score, the 3-bit file passed more hidden code tests in our Mac test) or Gemma 4 26B A4B for speed.

Lucy AI Studio sent free by Cephalon, which paid for an earlier Kingy AI video. Cephalon has no say in results. Mac and laptop bought by Kingy AI.

Who this page is for

People with a 32GB graphics card, and people with a 32GB Mac or shared-memory computer. By our fit rule the card can use 30.5 GB and the Mac 21.0 GB, so a 32GB Mac sits closer to a 24GB card than to a 32GB one.

32GB graphics card: top picks

Job Our pick File to download Score Speed
Everyday questions and writing Qwen3.6 35B A3B Ollama qwen3.6:35b (Q4_K_M) (21.72 GB) Kingy Lab everyday 99%, writing-to-rules 100% Speed not measured yet on a 32GB card; 62.0 tok/s on real jobs on our 128GB Lucy (Nov 14-15, 2026)
Long documents and files Qwen3.6 35B A3B Ollama qwen3.6:35b (Q4_K_M) (21.72 GB) long-document and file jobs 100.0; board 89.7 (likely 83.2 to 96.1) Speed not measured yet on a 32GB card
Coding and building apps Qwen3.8 27B or Gemma 4 31B Qwen3.8-27B-UD-IQ2_S.gguf (8.37 GB) or gemma-4-31B-it-Q4_K_M.gguf (18.69 GB) coding jobs: Qwen3.8 2-bit 98.1, Gemma 31B 94.6 (ranges overlap: a tie); Qwen3.6 86.5, lower but its range overlaps both Speed not measured yet on a 32GB card
Fastest good option Qwen3.6 35B A3B Ollama qwen3.6:35b (Q4_K_M) (21.72 GB) board 89.7 (likely 83.2 to 96.1) 62.0 tok/s on real jobs, the fastest of the 8 on our Lucy (Nov 14-15, 2026)

Why one model for almost everything: on our board the top five models that fit are close, and where ranges overlap we can’t separate them. Qwen3.6 35B A3B is in that group and is much faster than the dense models around it, because only about 3 billion of its 35 billion parameters work on each token (8 of its 256 experts plus a shared one, per Qwen’s model card). On our Lucy it wrote 62.0 tokens a second on real jobs; Gemma 4 31B managed 10.5 and Qwen3.8 27B’s 3-bit file 14.2. Those speeds are from the Lucy, not a 32GB card. Ollama also loads a 0.90 GB vision file with this model; with it the need is 25.07 GB, still well inside a 32GB card.

Everything we scored fits in a 32GB card’s memory except gpt-oss-120b, which needs 72.24 GB. With room to spare you can also run larger files of the same models (for example Gemma 4 31B’s 25.20 GB Q6_K), but we’ve only scored the files listed.

Model File we scored Memory needed, 8K context 32GB card (30.5 GB usable) Board score (on our Lucy)
Qwen3 4B Qwen3-4B-Q4_K_M.gguf (2.50 GB) 4.08 GB Fits, 26.4 GB spare 56.5
Qwen3 8B Qwen3-8B-Q4_K_M.gguf (5.03 GB) 6.86 GB Fits, 23.6 GB spare 74.5
Qwen3.8 27B, 2-bit file Qwen3.8-27B-UD-IQ2_S.gguf (8.37 GB) 9.80 GB Fits, 20.7 GB spare 97.9
Qwen3.8 27B, 3-bit file Qwen3.8-27B-UD-Q3_K_XL.gguf (13.15 GB) 15.05 GB Fits, 15.4 GB spare 93.1
Gemma 4 26B A4B gemma-4-26B-A4B-it-Q4_K_M.gguf (16.80 GB) 18.89 GB Fits, 11.6 GB spare 87.2
Gemma 4 31B gemma-4-31B-it-Q4_K_M.gguf (18.69 GB) 22.22 GB Fits, 8.3 GB spare 91.2
Qwen3.6 35B A3B Ollama qwen3.6:35b (Q4_K_M) (21.72 GB) 24.08 GB Fits, 6.4 GB spare 89.7
gpt-oss-120b gpt-oss-120b-F16.gguf (65.37 GB) 72.24 GB No: needs 72.24 GB, 41.7 GB over 98.0

32GB Mac: top picks

Job Our pick File to download Score Speed
Top board score Qwen3.8 27B, 2-bit file Qwen3.8-27B-UD-IQ2_S.gguf (8.37 GB) board 97.9 (likely 96.0 to 99.9) Speed not measured yet on a 32GB Mac
Most hidden code tests passed Qwen3.8 27B, 3-bit file Qwen3.8-27B-UD-Q3_K_XL.gguf (13.15 GB) board 93.1 (likely 83.5 to 100.0); Kingy Lab 97%; 13 of 14 hidden code tests on our 24GB Mac (2-bit: 10) Speed not measured yet on a 32GB Mac
Faster replies Gemma 4 26B A4B gemma-4-26B-A4B-it-Q4_K_M.gguf (16.80 GB) board 87.2 (likely 80.2 to 94.2); Kingy Lab 93% Speed not measured yet on a 32GB Mac; 54.5 tok/s on real jobs on our Lucy

Our rule counts two-thirds of a 32GB Mac (21.0 GB); macOS itself allows somewhat more. Gemma 4 26B A4B fits with 2.1 GB spare and Qwen3.8 27B’s 3-bit file with 5.9 GB. Gemma 4 31B (22.22 GB) and Qwen3.6 35B A3B (24.08 GB) are over. Some people raise the macOS limit to fit more; we haven’t tested that, and it leaves less memory for everything else.

Model File we scored Memory needed, 8K context 32GB Mac (21.0 GB usable) Board score (on our Lucy)
Qwen3 4B Qwen3-4B-Q4_K_M.gguf (2.50 GB) 4.08 GB Fits, 16.9 GB spare 56.5
Qwen3 8B Qwen3-8B-Q4_K_M.gguf (5.03 GB) 6.86 GB Fits, 14.1 GB spare 74.5
Qwen3.8 27B, 2-bit file Qwen3.8-27B-UD-IQ2_S.gguf (8.37 GB) 9.80 GB Fits, 11.2 GB spare 97.9
Qwen3.8 27B, 3-bit file Qwen3.8-27B-UD-Q3_K_XL.gguf (13.15 GB) 15.05 GB Fits, 5.9 GB spare 93.1
Gemma 4 26B A4B gemma-4-26B-A4B-it-Q4_K_M.gguf (16.80 GB) 18.89 GB Fits, 2.1 GB spare 87.2
Gemma 4 31B gemma-4-31B-it-Q4_K_M.gguf (18.69 GB) 22.22 GB No: needs 22.22 GB, 1.2 GB over 91.2
Qwen3.6 35B A3B Ollama qwen3.6:35b (Q4_K_M) (21.72 GB) 24.08 GB No: needs 24.08 GB, 3.1 GB over 89.7

Speed

We haven’t measured any model on a 32GB card or a 32GB Mac yet; we’ll add speeds here when we do. The chart shows board scores for the models that fit a 32GB card:

Bar chart of Kingy board scores for the seven models that fit a 32GB card, from Qwen3.8 27B 2-bit 97.9 down to Qwen3 4B 56.5

How to run it

The easy way: LM Studio. It has a built-in model downloader. Open the Discover tab (Cmd+2 on a Mac, Ctrl+2 on Windows or Linux), paste the repository name shown for each model below into the search bar, and pick the file with the same quant name (LM Studio’s guide).

The direct way: llama.cpp, the engine we test with. Install it with winget install llama.cpp on Windows or brew install llama.cpp on a Mac (install guide). -hf downloads a model from Hugging Face and -c 8192 sets the 8K context our fit numbers assume. -hf also downloads and loads a model’s vision file if it has one (about 1.2 GB for Gemma 4); our fit numbers are for text only, so the commands below add --no-mmproj to skip it (server options).

  • Qwen3.6 35B A3B: the file we scored is Ollama’s. Install Ollama, then run ollama run qwen3.6:35b. That tag is the same download as qwen3.6:latest on Ollama’s tag list (checked November 19, 2026).
  • Qwen3.8 27B, 3-bit file: Unsloth replaced many of its Qwen3.8 27B files, including both files we scored, on November 10 to 12, 2026 (commit history), so a plain download now gets a newer file than the one we scored. To use the exact file we tested, download Qwen3.8-27B-UD-Q3_K_XL.gguf from the August 20 revision, then run llama-server -m Qwen3.8-27B-UD-Q3_K_XL.gguf -c 8192. The newer file has the same name; we haven’t scored it.
  • Gemma 4 31B: llama-server -hf lmstudio-community/gemma-4-31B-it-GGUF:Q4_K_M -c 8192 --no-mmproj, or search lmstudio-community/gemma-4-31B-it-GGUF in LM Studio and pick Q4_K_M.
  • Gemma 4 26B A4B: llama-server -hf lmstudio-community/gemma-4-26B-A4B-it-GGUF:Q4_K_M -c 8192 --no-mmproj, or search lmstudio-community/gemma-4-26B-A4B-it-GGUF in LM Studio and pick Q4_K_M.

MLX on a Mac. Apple’s own MLX-LM runs models converted to MLX format: pip install mlx-lm, then mlx_lm.generate --model <an MLX model on Hugging Face> --prompt "hello" (or mlx_lm.chat for a chat session). MLX files are different downloads from the ones above, with different sizes, and we didn’t score or time them, so check the size before you download.

How we tested

Kingy tested every score on this page on our own machines. Nothing here is copied from a model maker’s chart.

  • Board score. Kingy’s local AI board v0.2 gives each model the same 9 real jobs: a landing page, a budget tracker app, a browser game, a Python bug fix, a chart from a messy spreadsheet, questions about a long document, a job that means searching 37 files with tools, a motion title and a video cut. Scripts check each result; no AI judges another AI. Every job runs 3 times, with the makers’ own settings and the exact files named on this page (checked by fingerprint). The board score is the average of 7 of those jobs, 21 runs in all. The landing page and the motion title are run but not counted, because they didn’t tell the models apart: every model on our Lucy scored 90.2 or more on the landing page, and motion title scores swung too much from one run to the next. The main runs were on our Lucy AI Studio (AMD Ryzen AI Max+ 395, 128 GB), November 14 to 15, 2026 (UTC). If a check script stopped early on a run, the checks it didn’t reach count as fails. “Likely range” is a rough 95% range around the average: when two ranges overlap, we treat the board as unable to tell those two models apart.
  • Writing and everyday scores come from Kingy Lab’s 120-task test (November 7 to 12, 2026), which marks 60 everyday jobs, 30 coding tasks and 30 writing tasks with rules to follow.
  • Fit uses the same rule as our Local AI checker: memory needed = file size + room for 8,192 tokens (8K) of conversation (the “KV cache”, from each model’s official settings file) + 10%. We count a graphics card’s memory minus 1.5 GB for the screen and driver, and, as a cautious rule of our own, two-thirds of a Mac’s memory up to 36 GB (three-quarters above that), leaving the rest for macOS and your apps. macOS itself lets the graphics side use more (about 19 GB of our 24GB MacBook Pro), so a Mac may fit a little more than we say. For Qwen3.6 and Qwen3.8 we leave out a small fixed memory state (under 0.2 GB). GB means a billion bytes. File sizes come from Hugging Face and Ollama, checked November 19, 2026. A “No” here means it doesn’t fit in that memory by this rule.
  • Speed is only shown where we measured it, with the chip and date. Everything else says “not measured yet”. We never estimate a speed on these pages. Speed on real jobs is how fast the model wrote during the 27 board runs: tokens written divided by the time spent writing, from llama.cpp’s own timings, counting every reply (including replies from runs we repeated after a dropped connection). It leaves out the time spent reading the prompt.

What we don’t know yet

  • Speeds and board scores on a 32GB card. Not measured yet.
  • Speeds on any 32GB Mac. Not measured yet.
  • Whether larger files of these models score higher than the ones we tested.
  • Image generation at 32 GB. Not measured, so no pick.

Not sure how much memory you have, or using a different chip? The Kingy Local AI checker asks for your machine and shows which of 50 popular models fit, with our measured speeds where we have them.

Sources

  • Kingy local AI board v0.2 and Kingy speed tests, checked 2026-11-19: Kingy’s board v0.2 scores and speed tests behind this page (method above).
  • Kingy Lab: Local AI Models Tested, checked 2026-11-19: Kingy Lab’s 120-task test of 8 open models on the Lucy, November 7 to 12, 2026 (writing, everyday and coding scores).
  • Kingy Local AI checker, checked 2026-11-19: The fit rule used on this page (Kingy verdict standard v0.1).
  • huggingface.co, checked 2026-11-19: Qwen’s Qwen3 4B files and sizes on Hugging Face.
  • huggingface.co, checked 2026-11-19: Qwen’s Qwen3 8B files and sizes on Hugging Face.
  • huggingface.co, checked 2026-11-19: Unsloth’s Qwen3.8 27B files at the revision we scored (August 20, 2026).
  • huggingface.co, checked 2026-11-19: Unsloth’s Qwen3.8 27B file history: many files, including both we scored, replaced November 10 to 12.
  • huggingface.co, checked 2026-11-19: Gemma 4 26B A4B files and sizes (LM Studio Community).
  • huggingface.co, checked 2026-11-19: Gemma 4 31B files and sizes (LM Studio Community).
  • huggingface.co, checked 2026-11-19: Gemma 4 31B settings file, used for the context-memory figure.
  • ollama.com, checked 2026-11-19: Ollama’s Qwen3.6 tags; the 35b tag is the file we scored.
  • huggingface.co, checked 2026-11-19: Qwen3.6 35B A3B settings file, used for the context-memory figure.
  • huggingface.co, checked 2026-11-19: Qwen’s Qwen3.6 35B A3B model card (total and active parameters).
  • huggingface.co, checked 2026-11-19: gpt-oss-120b file and size (Unsloth).
  • lmstudio.ai, checked 2026-11-19: LM Studio’s guide to downloading a model.
  • github.com, checked 2026-11-19: llama.cpp install guide.
  • github.com, checked 2026-11-19: llama.cpp server options (-hf, -m, -c).
  • github.com, checked 2026-11-19: Apple’s MLX-LM project.
  • ollama.com, checked 2026-11-19: Ollama download page.

Revision log

  • November 19, 2026: First published. Picks from Kingy board v0.2 (November 14 to 15) and Kingy Lab (November 7 to 12).