Cache Engineering for AI Agent Harnesses

An AI agent can be fast and wrong. A cache hit is not the same as a correct, authorised result.

Cache Engineering for AI Agent Harnesses is a systems-focused guide to designing, operating and evaluating reuse across LLM-powered applications. It treats caching not just as a speed optimisation, but as a contract between execution state, source data, access controls and production telemetry.

Across twelve chapters, readers explore prompt-prefix caching, transformer KV state, paged attention, quantisation and eviction; semantic caching, retrieval-augmented generation and vector-store caches; tool and API result reuse in multi-agent workflows; and the production challenges of tenant isolation, invalidation, freshness, provenance and cache poisoning.

Worked examples examine the cost of reuse alongside its correctness conditions. A recurring synthetic harness, Helix, links the chapters through reproducible Python laboratories, practical exercises and checked solutions maintained in a companion repository. The final chapters address cost and latency accounting, benchmarking, replay, observability and deployment decisions.

Written for AI engineers, ML infrastructure practitioners, software engineers and technical researchers, this book provides a disciplined framework for deciding when to reuse earlier computation, when to invalidate it and when to compute afresh.