If you don't run your own LLM proxy, how do you know whats going on?
- ID
- 37004
- Status
- summarized
- Published
- 19 Oct 2026, 5:09 AM
- Fetched
- 19 Oct 2026, 10:34 PM
- Provider
- Hacker News
- Category
- dev-community
- Original URL
- https://jrdsgl.com/if-you-dont-run-your-own-llm-proxy-how-the-fuck-do-you-even-know-whats-going-on/
- Source URL
- https://hnrss.org/best
Summary
- Score
- 7.5
- Created
- 19 Oct 2026, 10:34 PM
- Tags
- Audience
- developersvibe_codersai_ml_learnersai_agent_userssaas_founders
What happened
Jared Siegel argues that anyone building with LLMs should run their own proxy rather than relying on a paid frontend's analytics panel, which he calls unusable for engineering decisions. He lists five concrete reasons: one endpoint across multiple providers and models, logs of which key called which model with which client and at what time plus status, virtual API keys so your main key isn't scattered across work, agents and harnesses, infrastructure you control, and the ability to switch model or provider without re-plumbing everything. He also notes he has modified his own proxy for custom MCP server endpoints and better insight, and that pinpointing a cache miss during a coding session takes him about 20 seconds by hand because the proxy exposes an API.
Why it matters
If you are pasting one provider key into multiple agents, harnesses and side projects, you currently cannot answer 'which of these burned the tokens' or 'which one started failing' — request-level logs behind virtual keys answer that. Decide now whether your next agent project routes through a self-hosted proxy you can modify, because retrofitting visibility after the key is spread across five tools is the expensive version.
Discussion angle
What would you actually log per request to debug a bad agent run — model, key, client, latency, cache hit/miss, cost — and does your current setup capture any of it, or just a monthly token graph?