A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis
- ID
- 39327
- Status
- summarized
- Published
- 26 Oct 2026, 1:15 AM
- Fetched
- 26 Oct 2026, 4:08 AM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://lynchlab.net/llm-skin-cancer
- Source URL
- https://hnrss.org/best
Summary
- Score
- 8.0
- Created
- 26 Oct 2026, 4:08 AM
- Tags
- Audience
- developersai_ml_learnersai_agent_users
What happened
A dermatologist reports testing GPT-6 Astra and Claude Fable 5.1 against a fine-tuned 2015 Inception v3 CNN on 200 biopsy-proven skin lesions: the CNN named the correct diagnosis out of six for 125/200 (62%), while the two LLMs got 84 and 85 (42%). In one example, both LLMs called an invasive melanoma on the ear a seborrhoeic keratosis and said the patient could be discharged; the CNN called it melanoma.
Why it matters
For builders using LLMs for image classification or any high-stakes decision, this is a concrete reason to benchmark a small fine-tuned model before shipping a general LLM: a roughly 20-minute fine-tune beat two frontier models by 20 percentage points in this test, and the LLM error changed discharge advice. Do not treat a capable general LLM as a drop-in classifier without a task-specific baseline and an escalation path.
Discussion angle
When is a small fine-tuned model the better engineering choice than prompting a frontier LLM? Use the 62% vs 42% gap and the missed melanoma as a concrete case, then ask what would make this evaluation more trustworthy: more raters, prompt/version disclosure, or per-class breakdowns.