My local model setup on an M4 Pro Mac Mini
- ID
- 20763
- Status
- summarized
- Published
- 02 Sep 2026, 6:30 AM
- Fetched
- 04 Sep 2026, 1:53 AM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://lws.io/blog/my-local-model-setup/
- Source URL
- https://hnrss.org/best
Summary
- Score
- 7.0
- Created
- 04 Sep 2026, 2:59 AM
- Tags
- Audience
- developersai_ml_learnersai_agent_users
What happened
Kevin Lewis details his local LLM stack on an M4 Pro Mac mini with 48GB RAM, running Qwen3.6-35B-A3B-OptiQ-4bit for reasoning tasks and Gemma-4-E4B-it-OptiQ-4bit for lightweight chat, served via oMLX with Tailscale connecting his iPhone and MacBook. He replaced two $200/month cloud API subscriptions, citing cost unpredictability, silent model degradation, data privacy risk, and AI sovereignty concerns as motivations. Setup takes about 30 minutes and powers a Hermes agent backend accessible via Telegram.
Why it matters
If you are spending $200+/month on cloud LLM APIs and hitting rate limits or noticing quality drift, a 48GB M4 Pro Mac mini running quantized models via oMLX is a concrete break-even alternative worth pricing out. The specific model choices (Qwen3.6-35B-A3B-OptiQ-4bit, Gemma-4-E4B-it-OptiQ-4bit) and Tailscale-based remote access pattern are directly copyable for anyone who wants agent workflows without third-party API dependency.
Discussion angle
At what monthly API spend does local inference on Apple Silicon become the rational choice, and what are the real trade-offs in model quality versus the convenience and capability ceiling of cloud APIs?