TIME Is Serving AI Bots a Different Website, with Ads Built In
- ID
- 11812
- Status
- summarized
- Published
- 05 Aug 2026, 8:41 PM
- Fetched
- 07 Aug 2026, 5:39 PM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://www.vincentschmalbach.com/time-serves-ai-bots-a-different-website/
- Source URL
- https://hnrss.org/best
Summary
- Score
- 7.5
- Created
- 07 Aug 2026, 6:45 PM
- Tags
- Audience
- developersai_agent_userssaas_founders
What happened
TIME is serving AI crawlers a stripped-down markdown version of its site with sponsored content baked in, while humans get the full HTML page. The markdown responses include headers from ad-tech vendor Mobian tracking per-request impressions and token counts (e.g., x-mobian-tokens: 3323), with sponsored content like an Ally Bank FAQ embedded directly in list/section pages. TIME blocks some bots per-agent (GPTBot and ChatGPT-User get 406) while allowing others (ClaudeBot, PerplexityBot, OAI-SearchBot) through to the markdown.
Why it matters
If you build AI agents that fetch web content, publishers are now actively shaping what your agents see—injecting sponsored material into the markdown your LLM ingests and blocking certain bots entirely. This means agent-sourced answers may contain paid placements that are invisible to human users, and your bot's User-Agent determines whether you get content at all. Builders should test their agent's User-Agent against target sites and consider how sponsored-in-markdown affects answer trustworthiness.
Discussion angle
How should agent builders handle publishers injecting sponsored content into the markdown their LLMs consume—and should agent platforms disclose when fetched content contains paid placements?