27 billion parameters.
24GB of RAM. It fits.

It runs. Properly, not as a party trick.

Qwen3.8-27B is the model everyone is trying to run this week: a 27.8B-parameter dense model from Alibaba's Tongyi Lab, released 14 August 2026 under Apache 2.0. It takes text and images, has a native 262,144-token context window, and posts benchmark scores that trade blows with frontier API models on coding and agentic tasks.

The question we kept seeing: can you run it on a Mac with 24GB of unified memory, or do you need 32GB and up? So we did what we always do – picked a ready-provisioned machine from our own rack, in this case a Max-tier M4 Pro Mac mini with 24GB, and found out. Short answer: yes, at 4-bit quantisation, at around 11 tokens per second, with roughly 4GB to spare. But three settings decide whether it works or drowns, and none of them are defaults.

Pick the right GGUF, skip the rest.

The bartowski GGUF quantisations are the practical way to run this model in llama.cpp today. You need a build at least as new as b10419; we used b10488. On a 24GB Mac only two files are worth downloading:

• Q4_K_M (17.77GB) – the standard "good quality" 4-bit quant. Fits, but leaves the least headroom.
• IQ4_XS (15.57GB) – 2GB smaller, near-identical speed in our tests, and the extra headroom buys you context. This is the one we'd pick.
• Q5_K_M (20.75GB) and up – forget it on 24GB. The OS needs memory too.

Both quants ran within measurement noise of each other on speed, so the choice is purely about memory headroom:

Quant	File size	Prompt (pp2048)	Generation (tg128)
IQ4_XSpick this	15.57GB	95.4 tok/s	10.9 tok/s
Q4_K_M	17.77GB	96.8 tok/s	11.4 tok/s

Numbers are the mean of three llama-bench runs with flash attention on and every layer offloaded to the GPU. Run-to-run variance was under 3%.

macOS won't give the GPU 17.8GB by default.

On a 24GB Mac, macOS caps how much unified memory the GPU may wire. Load a 17.8GB model without touching that cap and llama.cpp either refuses to offload every layer or the machine starts paging. One sysctl fixes it:

# allow the GPU to wire up to 20GB (of 24GB total)
sudo sysctl iogpu.wired_limit_mb=20480

That leaves roughly 3.5GB for macOS, which is enough for a headless machine running an inference server, and it is not permanent: the setting resets to the default on reboot, so there is nothing to undo. After raising the limit, Metal reported a recommended working set of 21.5GB and the full model loaded onto the GPU cleanly.

The other two flags worth setting are flash attention and, if you want long context, a quantised KV cache:

# serve Qwen3.8-27B with an OpenAI-compatible API on port 8090
llama-server -m Qwen3.8-27B-IQ4_XS.gguf \
  -fa 1 -c 32768 -ctk q8_0 -ctv q8_0 --port 8090

With the IQ4_XS quant, flash attention, and an 8-bit KV cache, a full 32,768-token context loaded and answered fine on our machine: about 16.6GB resident with 20% of system memory still free. That is a quarter of a novel of context on a 24GB Mac mini.

Turn thinking off, or wait two and a half minutes.

Qwen3.8-27B is a reasoning model. Ask it for code with default settings and it thinks first: silently, at length, before a single character of your answer appears. In our first coding test it produced 6,500 characters of chain-of-thought and then hit our 1,600-token output cap without ever reaching the actual answer. At 11 tokens per second, that was 142 seconds of watching it deliberate.

On an API-hosted reasoning model you barely notice thinking time. At local speeds it dominates the experience, so for interactive use, turn it off. With llama.cpp's server it is one field in the request:

curl localhost:8090/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Write a Python CLI tool that watches a folder and converts new PNG files to WebP."}],
  "max_tokens": 1200,
  "chat_template_kwargs": {"enable_thinking": false}
}'

With thinking disabled, the same request came back in 28 seconds: a complete, correct 40-line Python tool using watchdog and Pillow, with install instructions, error handling, and a clean keyboard-interrupt shutdown. We ran it; it works. Keep thinking mode for hard problems you're happy to leave running, batch jobs, and overnight agent work, where the quality gain is real and the wait is irrelevant.

What 24GB buys you, and what it doesn't.

Our honest read: the 24GB M4 Pro mini is a genuinely usable home for this model at 4-bit. Eleven tokens per second is comfortable reading speed, prompt processing is quick, and the output quality at Q4 is strong. If you want one always-on machine that runs a frontier-class open model and everything else in your toolchain, it does the job at £129 per month.

But 27B dense at 4-bit is this machine's ceiling, and you can feel it. There is no room for the higher-precision quants where the model's benchmark scores actually live, no room for the multimodal projector next to a big context, and no room for the rest of your stack while the model is loaded. The unified-memory curve is unforgiving: the model that fits in 24GB with 4GB spare wants 32GB of headroom the moment you ask for more precision or more context.

How we ran these benchmarks.

Everything above was measured on a standard Max-tier machine from our rack (mini-M4P-590), not a tuned lab setup:

• Chip: Apple M4 Pro (12-core CPU, 16-core GPU)
• Memory: 24GB unified (shared between CPU and GPU)
• Storage: 512GB SSD
• OS: macOS 26.6.1
• Runtime: llama.cpp b10488, Metal backend, flash attention on
• Model: Qwen3.8-27B, bartowski GGUF quants (Q4_K_M and IQ4_XS)

Speed figures come from llama-bench (three repetitions per test) and from llama-server response timings for the end-to-end coding tasks. Tests were run on 18 August 2026, four days after the model's release. One nice aside: the 17.8GB model file downloaded in under four minutes on the rack's gigabit line.