Performance LRU Cache / Memoization See in Code Tour

LRU Caching & Performance

File state caching, memoized selectors, and prompt caching to minimize redundant I/O and API costs.

LRU Caching & Performance — Architecture Diagram

graph LR
    A[File Check] --> B{In LRU Cache?}
    B -->|Yes, fresh| C[Return cached metadata]
    B -->|No / Stale| D[Read filesystem]
    D --> E[Store in LRU]
    E --> C
    F[State Update] --> G[Memoized Selector]
    G --> H{Input changed?}
    H -->|No| I[Return cached result]
    H -->|Yes| J[Recompute + cache]

Mermaid diagram definition

Deep Dive

src/utils/fileStateCache.ts maintains an LRU cache of file metadata (existence, modification time). In a tool session that reads hundreds of files, this eliminates redundant stat() calls — each file is only checked once per session.

🔑Key Insight

Prompt caching is a different kind of caching: the Anthropic API caches large static sections of the system prompt (tool descriptions, memory content) across requests. This significantly reduces token costs on long sessions.

ℹ️Info

Memoized selectors in the state store are pure functions — given the same state slice, they always return the same result. They're re-computed only when their input slice changes, preventing cascading re-renders in the terminal UI.

KEY TAKEAWAYS

→LRU cache size should match session access patterns, not just memory limits
→Memoization only helps when inputs are referentially stable (immutable state helps)
→Prompt caching cuts API costs significantly on repeated system prompts
→Cache invalidation strategy: evict on session activity, not on time

Source Code

File state LRU cache — metadata caching with session-level eviction.

import { LRUCache } from 'lru-cache'
import { normalize } from 'path'

export type FileState = {
  content: string
  timestamp: number
  offset: number | undefined
  limit: number | undefined
  // True when this entry was populated by auto-injection (e.g. CLAUDE.md) and
  // the injected content did not match disk (stripped HTML comments, stripped
  // frontmatter, truncated MEMORY.md). The model has only seen a partial view;
  // Edit/Write must require an explicit Read first. `content` here holds the
  // RAW disk bytes (for getChangedFiles diffing), not what the model saw.
  isPartialView?: boolean
}

// Default max entries for read file state caches
export const READ_FILE_STATE_CACHE_SIZE = 100

// Default size limit for file state caches (25MB)
// This prevents unbounded memory growth from large file contents
const DEFAULT_MAX_CACHE_SIZE_BYTES = 25 * 1024 * 1024

/**
 * A file state cache that normalizes all path keys before access.
 * This ensures consistent cache hits regardless of whether callers pass
 * relative vs absolute paths with redundant segments (e.g. /foo/../bar)
 * or mixed path separators on Windows (/ vs \).
 */
export class FileStateCache {
  private cache: LRUCache<string, FileState>

  constructor(maxEntries: number, maxSizeBytes: number) {
    this.cache = new LRUCache<string, FileState>({
      max: maxEntries,
      maxSize: maxSizeBytes,
      sizeCalculation: value => Math.max(1, Buffer.byteLength(value.content)),
    })
  }

  get(key: string): FileState | undefined {
    return this.cache.get(normalize(key))
  }

  set(key: string, value: FileState): this {
    this.cache.set(normalize(key), value)
    return this
  }

  has(key: string): boolean {

AI Assistant

Ask anything about LRU Caching & Performance