Suppose you don’t purchase your AI coding agent as a subscription ( Anthropic and others ), but run it locally — on your own machine, on your own model. What does that mean, and what hardware do you need for that? We took a test on a MacBook Pro M5 Max (128 GB) as a starting point. Here is the data, hardware trade-off, and architecture that I, and you, might adopt.
The assumption
What if your coding agent runs locally by default, and only calls on the cloud when you explicitly ask it?
The figures below are measured on a MacBook Pro M5 Max with 128 GB RAM
Local AI models are standard, and immediately deliver three important things:
- Private. Your code does not leave your machine. No logging with a third party, no training data for someone else’s model, no changed conditions next year.
- Free. No per-token account. You run on hardware that you already have.
- Fast enough. Local is not “slow”. On the test M5 Max, the standard model achieves more than 140 tokens per second — we’ll get back to that.
What the test yielded — generation rate (tokens/second)
“Fast enough” does not display numbers. So we measured it. Six local models, provide the figures and context
| Model | tok/s | Type | Big one |
|---|---|---|---|
| qwen3 coder (default) | 140 | MoE, 3B active | 18GB |
| qwen3.5:9b | 74 | dense 9B | 6.6GB |
| gemma3:12b | 61 | dense 12B | 8.1GB |
| qwen2.5:14b | 58 | dense 14B | 9.0GB |
| qwen3:14b | 48 | dense 14B (reasoning) | 9.3GB |
| qwen3:32b | 24 | dense 32B (reasoning) | 20GB |
Two things stand out. One: big doesn’t automatically mean fast. qwen3-coder is by far the fastest — 140 tok/s — while being 18 GB in size. It’s one mixture-of-expertsmodel (MoE) of which only 3 billion parameters are active per token. You load a big brain, but think with a small, fast part. Exactly the sweet spot for code. Two: the heavy reasoning model is slow. qwen3:32b achieves 24 tokens/s — 16 seconds for 400 tokens. That’s why you don’t use it by default, but only call it when you want heavy offline analysis.
The routing architecture
One agent, two layers:
pi (lokaal)
└─ defaultProvider: ollama → http://127.0.0.1:11434
├─ lokale tags → Metal-GPU op de Mac
└─ :cloud tags → cloud (proxy via de lokale daemon)The smart choice is in the Standard choice: not the largest model, but the model that has the best balance for 95% of the work. Here it is qwen3-coder — fast, capable, local. The heavy models are in the cloud and you can only call them via /model on. No automatic cloud call “just to be sure”. You consciously choose when you use them.
What hardware do you need for this?
The test was on a MacBook Pro M5 Max (128 GB). For Brainfusion there are two realistic options to run LLMs locally:
- MacBook Pro 16″ M5 Max, 128 GB, 40-core GPU — 614 GB/s bandwidth, 128 GB memory. Portable, runs 70B-parameter models entirely in memory. Only Mac with more than 96 GB now (the Mac Studio 256/512 GB has been discontinued due to a global DRAM shortage). About €8,300.
- Mac Studio M3 Ultra, 96 GB, 60-core GPU — 819 GB/s bandwidth (fastest Mac), 96 GB memory, runs a 70B comfortably at 25–30 tok/s. Always on desktop, quiet, low power (100–200 W) — ideal as a 24/7 inference server. About €6,800.
The rule for local LLMs: memory bandwidth > raw compute for tokens per second. RAM determines whether a model loads; bandwidth determines how fast it runs. The M5 Max gives the most capacity (128 GB) plus it’s a laptop so you always take it with you.; the Mac Studio gives the most bandwidth (819 GB/s) plus it is quiet and always on and can handle heavy tasks locally for web server services. Together they can support a developer and the standard services.
What you learn from it
- Measure, don’t guess. “Local is too slow” is an opinion. A benchmark shows that the standard model achieves 140 tok/s — more than enough for most coding work.
- MoE wins locally. A mixture-of-experts with few active parameters is the right architecture: large enough to be capable and small enough to use to be fast.
- Standard = your main choice. Not the strongest model, but the one that covers most tasks without you even thinking about it. Escalation is a conscious action, not automatic.
- Cloud is not treason. It’s a tool. As long as the Standard is local and the cloud when necessary, you retain control and privacy for the vast majority of your work.
And now?
Running locally is mature. You get speed on a modern Mac with sufficient memory, with a standard model that is fast and capable, and an escalation path to the cloud for times when local falls short. No per-token account. No code that goes out your door. No dependence on a third party.
That is ownership instead of rent — the principle on which all of Brainfusion runs. We are therefore considering our own hardware (an M5 Max and a Mac Studio) to further automate and improve our services via AI. We want to involve donors in this; a fundraiser will follow in a next blog page. Would you like to think along or start a conversation? Let us hear from you.
The benchmark numbers were measured on a MacBook Pro M5 Max (128 GB, 18-core CPU, 40-core GPU) as a test. The script provided averages over multiple measurements. Hardware prices are approximate (Apple NL, July 2026)