An AI assistant for this site, end to end
A chatbot that answers questions about the site and about me, and acts on the page. Built with guardrails, a budget and a test set, run from a private dashboard, and set up to improve from real use.
- 40questions in its test set
- $0.002per message at launch, with a warm cache
- $0.80a day, the budget for the whole site
Problem
A site about someone who builds with AI, with no AI on it, would feel weird. I also wanted to show that I can set up an AI tool properly: not a demo that works once, but a product with limits, tests and a plan for improving it.
It had to answer questions about the site and about me, act on the page (switch the theme, point to the right section, prepare a message in the contact form), stay within a fixed budget and keep bots out. And it had to do that without a single third-party script in the browser.
Approach
An interview, then a plan
Twelve questions, each with a recommended answer, settled the choices before any code: the model, the budget, what it may say, what it keeps and for how long. I designed the interface in Claude Design; Claude Code built it from the plan.
One model, called only by the server
The assistant runs on Claude Sonnet 5.5. The browser only talks to the site itself, so the site still loads nothing from other companies. Its instructions, the site's own content and its tools form a fixed prefix (about 6,400 tokens at launch) that's cached, so most messages only pay for the new part.
Five tools, checked by the server
It can switch the theme, offer links to known pages and sections, show the contact options, prepare a message for the contact form and suggest a next question. Each tool has a strict schema, and the server checks every call before the browser carries it out. It can never link to a page that doesn't exist.
Guardrails
A proof-of-work check in the browser before the first message, written for the site so it needs no third party. Requests only from the site itself. Limits of 15 messages per conversation, 30 per visitor per day and 1,000 characters per message, and a budget of $0.80 a day for the whole site, under a monthly limit at Anthropic. Emails and phone numbers are removed before anything is stored, visitors appear only as a code that changes every day, and conversations are deleted after 7 days, as the privacy notice says.
A test set before launch
40 questions: facts from every page, questions it should say it doesn't know, topics it must leave alone, attempts to make it sell or change its rules, Dutch questions, and every action. Each answer is checked automatically for words that sound like selling, prices, invented links and the mistakes below. It runs against the real model by hand, never in CI, and must stay under $0.50 a run. The first run passed 40 of its 41 checks for $0.23; I fixed the instructions, and the second passed everything.
What went wrong after launch
- It pointed at buttons it couldn't see. It said "tap the question above", while the button appears below its answer. The model got another turn after each tool call with nothing to go on, and guessed. The instructions now tell it what a tool shows and where, and the test set fails any answer that says "above", "below", "tap" or "click".
- It repeated itself. After writing a full answer and calling a tool, it got another turn "to finish" and said the same sentence again. The server now only gives it another turn when it called a tool before writing anything. That also saves a request on every answer with a link: a full test run went from $0.23 to $0.16. The test set now fails any answer with two near-identical sentences.
Running it
A private dashboard runs it: a switch, today's numbers (counts only, never what visitors wrote), its limits, a history of every change and the devices that can sign in. It sits behind a hidden link and passkeys, so there's no login page to attack. A change applies at once, without a deploy.
Result
The assistant has been live since 29 October 2026. It answers in the visitor's language, prepares messages for the contact form and stays within its budget, at about $0.002 per message with a warm cache at launch.
The log keeps conversations for 7 days, long enough for a weekly review, and that's the next step: a stronger model reads the week's conversations and finds wrong answers, questions it couldn't answer and misuse. I decide what changes, and every wrong answer becomes a new question in the test set, so the same mistake is caught from then on.
