Write the read-cache contract first: even a 5-second session cache must not extend auth
The useful part of this performance patch was not the HashMap. It was the contract: three read models, separate TTLs, field allowlists, synchronous invalidation, generation checks against late inserts, and tests for revocation and expiry before celebrating the cache hit.
The problem
“Add a cache” is too vague for authenticated product code. A cache can remove database trips, but it can also keep a revoked cookie alive, show a stale profile, or accidentally expose a field that the original query never meant to return.
The patch behind this post added an in-process read cache to BluffKing's server. The important part is not the HashMap. The important part is the contract around it: what is allowed to be stale, for how long, what mutation invalidates it, and which fields are allowed to enter the value at all.
The code says that boundary out loud. server/src/read_cache.rs:1-14 declares that this is not a generic HTTP response cache; it only covers three low-volatility read models: a positive session identity, the caller's /api/auth/me payload, and public-at-the-table avatar/profile fields.
Start with the read contract
The reusable move is to write the cache table before writing the cache code. In this patch, each cache has a separate stale window and a separate safety rule:
| Read model | Allowed window | Safety rule |
|---|---|---|
| positive session lookup | 5 seconds | Revocation must evict synchronously, and the cache may never extend the session's real expiry. |
/api/auth/me | 5 seconds | Profile, email, and session mutations invalidate the user read model; membership/entitlement changes rely on the safety TTL. |
| avatar/profile fields for table seats | 60 seconds | The value contains only id, display name, avatar key, and avatar data — no email. |
Those numbers are not prose. They are constants in server/src/read_cache.rs:28-37. The avatar field allowlist is a concrete struct at server/src/read_cache.rs:77-84, and the handler repeats the same boundary before querying: server/src/handlers/users.rs:1244-1254 selects only id, display_name, avatar_key, and avatar_data.
That is the difference between caching a query and caching a contract. A query says “this SQL was expensive.” A contract says “this is the only shape we are willing to serve from memory.”
Invalidate before you celebrate the hit rate
The session cache is the dangerous one. It caches only positive identity lookups, so a stale hit would authenticate a cookie that should be dead. The patch closes that in two ways.
First, the session lookup returns a cache_valid_until value capped by the real database deadlines. server/src/auth/session.rs:219-275 checks the absolute expires_at, checks the idle timeout, and gives the cache the earlier of those deadlines. Then server/src/read_cache.rs:333-355 inserts with a TTL no longer than that remaining validity.
Second, logout and logout-all push invalidation into memory immediately. The WebSocket revocation registry calls invalidate_session_hash_all_pools for one token and invalidate_user_all_pools for a whole user at server/src/ws_revocation.rs:223-240. The cache implementation removes matching entries and bumps a generation counter at server/src/read_cache.rs:412-453.
The generation counter handles the race most cache patches forget: miss the cache, start a database read, receive a revoke, then insert the now-stale value after the await. A load snapshots the generation before I/O, and put_session_if_current / put_me_if_current / put_avatar_if_current insert only if that generation is still current (server/src/read_cache.rs:270-273, 323-390, 397-409).
Prove the failure modes, not just the fast path
The regression tests are where the contract becomes real. They do not merely assert that a second request is fast.
server/tests/avatar_data.rs:168-182primes/api/auth/me, writes an avatar, and asserts the cached/mevalue was invalidated before the next read.server/tests/avatar_data.rs:193-224checks overlapping avatar batches collapse to one per-user result while preserving the no-email field boundary.server/tests/avatar_data.rs:226-237proves logout evicts the positive session cache and the same token returns unauthorized immediately.server/tests/avatar_data.rs:239-261proves the five-second session cache cannot extend a session whose database expiry is only 150 milliseconds away.server/src/read_cache.rs:512-580tests the late-fill-after-invalidation race directly, without needing a live query.
There is a performance guard too: server/tests/avatar_data.rs:264-309 runs 100 warm in-process calls each for /api/auth/me and /api/users/avatars, then keeps p95 under 5,000 microseconds. That is a service-path regression guard, not a production latency claim; it excludes TLS, network transit, and the real host's load.
The part to copy
When I let an agent add a read cache, I do not ask for “a cache.” I ask for the contract:
- Name the exact read models allowed to be cached.
- Give each model its own TTL and memory bound.
- Write the field allowlist into the cached value type.
- Wire every mutation that can make the value false before counting the speedup.
- Protect the miss → await → stale insert race with a generation or version check.
- Test revocation, expiry, mutation, and field secrecy as first-class cases.
A cache without that list is only a faster database query. A cache with that list is a read-path contract: the system can be faster because it knows exactly what truth it is allowed to reuse.