Jev meets the Trolley Problem
- ID
- 36145
- Status
- summarized
- Published
- 16 Oct 2026, 12:24 AM
- Fetched
- 16 Oct 2026, 10:16 AM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://zachlandes.com/trolley-problem/
- Source URL
- https://hnrss.org/best
Summary
- Score
- 7.5
- Created
- 16 Oct 2026, 10:16 AM
- Tags
- Audience
- developersai_ml_learnersai_agent_usersvibe_coders
What happened
Zach Landes benchmarked a new class of 'decision models' — models that output choices with attached confidence probabilities instead of text — on the Trolley Problem, running Jev, OpenAI's Decisions API (Luna), eleven local models chosen by popularity or launch coverage (including Clef, Clef-flash and Strands), plus Laya as an untuned baseline, across 5,240 setups and repeat runs. The post describes these models as fast, cheap and general, claiming they can be dozens or even hundreds of times cheaper and faster than an LLM like Claude while purportedly matching a lower-tier frontier model's intelligence. The excerpt cuts off before any results, so the actual per-model answers and confidence numbers are not in the text.
Why it matters
If the 'dozens to hundreds of times cheaper' claim holds, the LLM call you use for routing, triage or yes/no gates inside an agent loop is the most obvious thing to swap — and since the twelve local models in this comparison are ones runnable on a Mac, that swap can also move inference off metered API spend. But the excerpt ends before the results, so don't switch anything yet: read the post for the per-model verdicts and probability calibration first, because a decision model that returns confident wrong choices is worse than a slow LLM you can read.
Discussion angle
The 5,240-setup methodology is the real substance here: how would you decide that a decision model's probability output is calibrated enough to replace an LLM call in your own agent loop, and what does it cost you when it silently routes the wrong way with high confidence?