Llm

Part 1 was for the person who signs the invoice. This one is for the person who gets handed the Jira ticket afterward. If you read that post and thought “yes, fine, I want this, how hard can it be” — this post is the honest answer to that question. It’s a deployment walkthrough with the bad parts left in. The code is at github.com/kenkitts/claude-apps-gateway-cdk. Five Python CDK stacks, a Docker image that fetches and verifies the claude binary at build time, and enough inline comments to reconstruct most of the decisions made under pressure. Clone it now if you want to follow along; the scars will make more sense with the code open. ...

Your developers are already using Claude Code. Right now. On personal Anthropic accounts or individual API keys, expensed on a T&E line that says “software tools,” with your source code as the payload. You can’t see the spend, can’t say who’s spending it, can’t cap it, and if you asked your CISO whether any of this is compliant with your data-handling policy, she would say something unprintable. That is the status quo an LLM gateway is reacting to. Not a hypothetical future where AI tools proliferate — the present, where they already have. ...

In Part 2 we gave the model hands — tools that do things. In Part 4 we gave it longer arms — tools that do things on someone else’s machine. Both answered the same question: how does the model act? This post answers a different one: how does the model know how? Not facts. Facts are RAG — go fetch a document, stuff it in the context, done. I mean procedure. The gap between a junior who can write code and a senior who knows how to run a code review isn’t more data. It’s knowing the steps, the order, the things to check, the traps. You can’t bolt that on with a vector search. It’s a recipe, not an ingredient. ...

In Part 1 we built the loop. In Part 2 we handed the model some knives — tools, sandboxed so it couldn’t read your SSH keys. Both parts leaned on a thing I kept gesturing at and refusing to explain: the Session object. The agent’s memory. Time to explain it. Here’s the uncomfortable foundation everything in this post sits on: The model remembers nothing. Each call to Bedrock is a blank slate. The model that answered your last question has no idea it ever existed. It is a brilliant amnesiac, waking up fresh every single time, and the only reason it appears to “remember” the conversation is that you shove the entire transcript back in front of it on every request. Memory isn’t a model feature. It’s a thing your harness does, by hand, at a cost. And the bill — we’ll get to the bill. ...

In Part 2 we forged our own knives: a calculator that parses an AST instead of calling eval, file tools locked in a sandbox, every input frisked by code we wrote and could actually read. We trusted those tools because we built them. This post is about handing the model a knife from a stranger’s kitchen, sight unseen, and saying “go nuts.” That’s the Model Context Protocol, or MCP. It lets a tool live on someone else’s server, behind a login, and still show up on the model’s menu like it grew there. And here’s the part that should make the back of your neck prickle: ...

In Part 1 we established the simple truth: LLMs can only produce text. The harness is the babysitter that turns that text into action. Today we build the action part. The tools. The hands. This is the fun chapter, by which I mean it’s the one where we voluntarily give a probabilistic text generator the ability to touch the filesystem and then act surprised when it tries to read /etc/passwd. All the code below is lifted straight from the repo — a single agent.py file you can clone and run. Strap in. ...

You’ve seen the demos. AI agents booking flights, writing code, querying databases, probably filing your taxes while whispering sycophantic flattery. Very impressive. Very “the future is here, and it’s kinda scary.” Here’s the simple secret nobody mentions in the keynote: LLMs can only produce text. That’s it. They can’t run code. They can’t check the time. They can’t read a file. They are the world’s most confident interns — full of opinions, zero ability to actually do anything. All they do is predict the next token and hope you’re impressed. ...