Build your own decision model
- ID
- 34163
- Status
- summarized
- Published
- 11 Oct 2026, 6:50 AM
- Fetched
- 11 Oct 2026, 8:19 PM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://nishtahir.com/build-your-own-decision-model/
- Source URL
- https://hnrss.org/best
Summary
- Score
- 6.5
- Created
- 11 Oct 2026, 8:19 PM
- Tags
- Audience
- developersvibe_codersai_ml_learnersai_agent_users
What happened
Nish Tahir's post contrasts "system one" decision models (e.g. Jev) that constrain output to a fixed option set and answer in a single forward pass, with standard LLM structured output where the model needs a decode pass per token (his example needs 11 passes for JSON). It then walks through emulating the decision-model pattern yourself in ~40 lines of Python: load Qwen/Qwen3-1.7B with transformers, grab the logits at the final position, and restrict/read probabilities only over the first-token IDs for options A through E. He flags a caveat: the resulting token probabilities likely reflect confidence in the next token, not the true probability the selected answer is correct, unless you do additional training. The Hacker News thread drew 335 points and 75 comments.
Why it matters
If you currently burn a full generation (and multiple decode passes) just to pick one of a handful of labels, this shows a single-forward-pass alternative using a 1.7B local model and masked logits over option token IDs. The concrete decision to make: use it for cheap classification/routing where a wrong label is recoverable, but do not expose the softmax value as a confidence or correctness score without calibrating, since the author explicitly says those numbers are not the probability of being right.
Discussion angle
Where does single-pass constrained classification actually beat a small fine-tuned classifier or a full JSON-mode call in your stack — and what would it take to turn those logit probabilities into numbers you'd trust for routing decisions?