Reasoning at 34 tok/s.
Chain-of-thought. On device.

Smaller models are getting smarter — and faster.

This is the second round of benchmarks on our Pro-tier M4 Mac mini. Last time we tested Qwen 3 8B, Llama 3.1, and Qwen 2.5 against GPT-4o. This round focuses on what's new: Alibaba's Qwen 3.5 family, a dedicated reasoning model with chain-of-thought, and OpenAI's first open-weight release.

The standout is Qwen 3 4B Thinking — a reasoning model that works through problems step-by-step before answering. At just 4B parameters it generates at 30–34 tok/s on the M4, faster than the 8B models we tested last time, while producing noticeably better output on complex tasks.

Binary search: explain and implement.

Same prompt as our first benchmark: write a Python binary search function and explain each step. This lets us compare directly with previous results.

Model	Generation	Tokens
Qwen 3 4B Thinkingreasoning	30.0 tok/s	5,126
Qwen 3.5 4Bnew	18.2 tok/s	4,372
Qwen 3.5 9Bnew	13.3 tok/s	3,022
GPT-OSS 20Bnew	9.2 tok/s	2,211
Qwen 2.5 7Bbaseline	22.3 tok/s	788
Llama 3.1 8Bbaseline	21.0 tok/s	365
Qwen 3 8Bbaseline	19.5 tok/s	1,509

The thinking model generated 5,126 tokens — far more than any other model — because it includes its chain-of-thought reasoning in the output. Despite producing 3× more tokens than most models, it finished faster thanks to its 30 tok/s generation speed.

Qwen 3.5 4B slots in at 18.2 tok/s, slightly below the previous-gen 8B models. Qwen 3.5 9B runs at 13.3 tok/s — slower than the 7–8B class, but early reports suggest notably better output quality for the trade-off.

Build a full REST API from scratch.

Same production-grade coding task: write a complete Express.js REST API with five CRUD endpoints, Zod input validation, error handling middleware, and TypeScript types.

Model	Generation	Tokens
Qwen 3 4B Thinkingreasoning	33.9 tok/s	2,713
Qwen 3.5 4Bnew	19.0 tok/s	2,510
Qwen 3.5 9Bnew	12.8 tok/s	4,079
GPT-OSS 20Bnew	9.2 tok/s	5,338
Qwen 2.5 7Bbaseline	22.3 tok/s	1,399
Llama 3.1 8Bbaseline	20.8 tok/s	1,335
Qwen 3 8Bbaseline	19.4 tok/s	2,048

GPT-OSS 20B is interesting. It's a mixture-of-experts architecture — 21B total parameters with only 3.6B active per token — quantised to MXFP4 so it fits in 16GB. At 9.2 tok/s it's the slowest model we've tested, but it produced 5,338 tokens of thorough, well-structured output. For batch tasks where speed matters less than quality, it's a viable local option.

The thinking model hit 33.9 tok/s on the coding task, its fastest result. Its chain-of-thought approach meant it reasoned through the API design before writing code, producing cleaner error handling and more consistent validation patterns than models that jumped straight to output.

Every model we've tested on M4.

Combined results from both rounds of benchmarks, ordered by average generation speed. All local models run on the same Pro-tier Mac mini via Ollama.

Model	Params	Avg tok/s	Notes
GPT-4oapi	n/a	142.5	Cloud API baseline
DeepSeek Coder V2 16B	16B	51.5	Fastest code model
Qwen 3 4B Thinkingreasoning	4B	32.0	Chain-of-thought
Qwen 2.5 7B	7B	22.3	Fastest 7B class
Llama 3.1 8B	8B	20.9	Solid all-rounder
Qwen 3 8B	8B	19.5	Built-in reasoning mode
Qwen 3.5 4Bnew	4B	18.6	Next-gen small model
Qwen 3.5 9Bnew	9B	13.1	Quality over speed
DeepSeek R1 14B	14B	11.5	Reasoning model
GPT-OSS 20Bnew	20B (MoE)	9.2	OpenAI open-weight

Reasoning models are the real story.

Six months ago, chain-of-thought reasoning was exclusive to cloud APIs like o1 and o3. Now you can run it locally at 34 tok/s on a £89/mo Mac mini. The thinking model produced consistently better code structure, clearer explanations, and more robust error handling than non-reasoning models of the same size.

GPT-OSS marks OpenAI's first real open-weight release. It's a mixture-of-experts model — 21B total parameters but only 3.6B active per forward pass. On the M4 it runs at 9.2 tok/s, making it best suited for background tasks, batch processing, or cases where you specifically want an OpenAI-derived model running locally.

How we ran these benchmarks.

All benchmarks were run on the same Halfpenny Mac Pro-tier machine (mini-02-524) used in our first benchmark:

• Chip: Apple M4 (10-core CPU, 10-core GPU)
• Memory: 16GB unified (shared between CPU and GPU)
• Storage: 512GB SSD
• Runtime: Ollama on macOS Sequoia
• New models: Qwen 3.5 9B, Qwen 3.5 4B, Qwen 3 4B Thinking 2507, GPT-OSS 20B (MXFP4)

Each model was tested twice with identical prompts using the Ollama HTTP API with stream: false. Generation speed is calculated from the API response metadata. Tests were run on 31 May 2026.