← Back to blog

·

The website test that re-reads read_cache.rs before a cache claim ships

A file:line citation is only a bookmark until something checks it. We turned one cache article's TTLs, field allowlist, invalidation path, and latency caveat into a website test, so drifted public claims fail before publishing.

Editor's note, 2026-09-05. The fifth checklist item below used to cite a candidate-mining script, scripts/blog/blog-candidates.mjs, and the dedupe check inside it. Commit e836ad28 deleted that script along with the rest of the development harness, and nothing replaced it, so the item has been rewritten to drop the claim rather than keep a citation that resolves to nothing. The sourceRange() helper and the source-reading tests described below still live in website/tests/posts.test.mjs. Two sentences about the surrounding process were corrected in the same pass — the standing cross-vendor review, and this test acting as a publish gate. Neither is enforced the way the original text implied.

The problem

AI-generated technical writing has a quiet failure mode: the prose can stay confident after the code moves. A file:line citation helps, but if nobody checks it, it is only a bookmark. The next patch can change a TTL, widen a returned field, or move an invalidation path while the public article still says the old thing.

The branch commit ab6b277b6fb5fc4689826ee218a442c008678fc0, merged by bd78108af3ab1dc0a24f5c6e821afbc602a631d4 (Merge feat/blog-fairness-fail-closed), published the read-cache contract article and added tests for the article in the same branch.

The reusable lesson is narrower than “test every sentence.” When a public technical post makes a claim with trust, security, or product meaning, make that claim executable before publishing it.

Start with the dangerous claims

The cache post says specific things that would be expensive to get wrong: the session and /api/auth/me caches use 5-second windows, the avatar/profile cache uses 60 seconds, cached avatar data must not include email, and the warm-path microsecond budget is not a production latency promise.

So the first test names those promises directly. In commit ab6b277b6fb5fc4689826ee218a442c008678fc0, the test named pins the cache windows, invalidation boundary, field allowlist, and latency disclaimer reads both language bodies for cache-the-read-path-contract and asserts that the article still carries the TTLs, the no-email boundary, the membership/entitlement TTL caveat, and the latency disclaimer. It also rejects two older wrong phrasings, so a reviewer cannot accidentally restore an overclaim while rewording the post.

That is only the first layer. A test that only reads the article can prove that the prose stayed consistent with itself; it cannot prove the prose stayed consistent with the code.

Then read the code from the content test

The useful part is the sourceRange() helper added to website/tests/posts.test.mjs in commit ab6b277b6fb5fc4689826ee218a442c008678fc0. It reads a repository file and slices the exact line range the article cites. The follow-up test named keeps the cited Rust source ranges aligned with the cache contract does not trust the blog body. It opens the Rust source and checks for the real tokens: SESSION_TTL, ME_TTL, AVATAR_TTL, CachedAvatar without email, generation-based late-insert protection, session invalidation, avatar query fields, and the warm-path p95 assertions.

Now the article has a fail-closed path. If a future patch turns 5 seconds into 30, adds email to the cached avatar type, removes the generation guard, or turns a service-path regression budget into a production latency claim, the website test goes red — it reads the source, so the claim and the code cannot part company unnoticed. It is a repository test rather than a CI gate, so what it protects is whoever runs the site's tests before shipping.

What this does not prove

This pattern is a tripwire, not a reviewer. It does not judge whether the English is sharp, whether the Chinese is native, whether the article picked the best lesson, or whether every possible claim was covered. It also does not make a production performance promise true; the cache article still labels the 5,000-microsecond guard as an in-process service-path budget, not a live-host number.

The point is to separate two jobs. Judgement — clarity, framing, overclaim — needs a reader, a person or another model. The content test handles the claims a machine can check: source ranges, constants, field boundaries, and phrases that must not come back.

The part to copy

For AI-written public engineering content, I now use this checklist:

  1. Pick the three to five claims that would damage trust if they drifted.
  2. Put a concrete file:line source beside each claim.
  3. Add a content test that reads the cited source range, not just the article text.
  4. Assert the dangerous negative too: the field that must not appear, the phrase review retired, the latency label that must stay scoped.
  5. Record the merge the post was written from in the registry's sourceShas[] field (website/lib/posts.ts), so the next author can see which shipped change a lesson has already been drawn from. It is authoring provenance, not a published field: nothing on the site renders it, and the content test asserts the field and this article's own entry are still there.

A technical post should not get the benefit of executable evidence only after a reader catches it. If the claim matters, turn it into a test before it reaches the public site.