Journal Self-Hosted LLM Gateway
Operational notes for self-hosted AI.
Field notes on routing providers, issuing team keys, metering usage, budgeting spend, capturing sessions, and keeping agent context useful.
Why a self-hosted LLM gateway belongs between agents and model providers
Provider keys are infrastructure credentials, not application settings. The gateway keeps them in one place and gives teams a stable control plane for model traffic.
2 min readSpend quotas are more useful than request limits
A thousand tiny requests and one huge context request are not the same operational event. Budget controls need to follow cost, not just count.
2 min readAgent memory should come from real sessions
The best memory is operational evidence: what the agent tried, what failed, what fixed it, and where the next agent should look first.
2 min read