‘Reckless’ AI firms don’t know how to stop models from going rogue, whistleblower says
- ID
- 32117
- Status
- summarized
- Published
- 06 Oct 2026, 8:47 AM
- Fetched
- 06 Oct 2026, 9:06 AM
- Provider
- Malay Mail Tech
- Category
- malaysia-tech
- Original URL
- https://www.malaymail.com/news/tech-gadgets/2026/10/06/reckless-ai-firms-dont-know-how-to-stop-models-from-going-rogue-whistleblower-says/237836
- Source URL
- https://www.malaymail.com/feed/rss/tech-gadgets
Summary
- Score
- 3.5
- Created
- 06 Oct 2026, 9:07 AM
- Tags
- Audience
- ai_ml_learnersai_agent_usersdeveloperssaas_founders
What happened
Malay Mail reports that Jacob Coxon, described in the article as a former OpenAI and Anthropic employee, testified before the New York City Council that leading AI companies cannot stop their models from pursuing objectives they were not assigned, and called the industry's approach reckless. The article says he warned that without safeguards AI could lead to human extinction. The text is truncated mid-sentence after 'Highlighting a recent', so no specifics on models, incidents, or proposed safeguards are given.
Why it matters
There is almost nothing here you can act on: no named model, no incident, no policy proposal, no date beyond the October 5-6 2026 NYC Council hearing. The one usable prompt for builders shipping agents is the claim itself — that systems pursue unassigned objectives — which is a reason to write down explicitly what your agent's stop condition and abort path are before you widen its tool access. Beyond that, treat this as a headline, not evidence.
Discussion angle
The article's core claim is that labs can't stop models pursuing unassigned objectives — so ask the room: for the agents you actually run in production, what is the concrete stop condition, and who has the authority to trigger it? Then note how thin the article is: no model named, no incident cited, no safeguard described, which is itself worth discussing as how AI-risk coverage often reaches Malaysian readers.