Chapter 2 System Design

Query Engine & Main Loop

The pipeline that turns a user message into Claude's response

src/query.tsLines 160
1
// biome-ignore-all assist/source/organizeImports: ANT-ONLY import markers must not be reordered
2
import type {
3
  ToolResultBlockParam,
4
  ToolUseBlock,
5
} from '@anthropic-ai/sdk/resources/index.mjs'
6
import type { CanUseToolFn } from './hooks/useCanUseTool.js'
7
import { FallbackTriggeredError } from './services/api/withRetry.js'
8
import {
9
  calculateTokenWarningState,
10
  isAutoCompactEnabled,
11
  type AutoCompactTrackingState,
12
} from './services/compact/autoCompact.js'
13
import { buildPostCompactMessages } from './services/compact/compact.js'
14
/* eslint-disable @typescript-eslint/no-require-imports */
15
const reactiveCompact = feature('REACTIVE_COMPACT')
16
  ? (require('./services/compact/reactiveCompact.js') as typeof import('./services/compact/reactiveCompact.js'))
17
  : null
18
const contextCollapse = feature('CONTEXT_COLLAPSE')
19
  ? (require('./services/contextCollapse/index.js') as typeof import('./services/contextCollapse/index.js'))
20
  : null
21
/* eslint-enable @typescript-eslint/no-require-imports */
22
import {
23
  logEvent,
24
  type AnalyticsMetadata_I_VERIFIED_THIS_IS_NOT_CODE_OR_FILEPATHS,
25
} from 'src/services/analytics/index.js'
26
import { ImageSizeError } from './utils/imageValidation.js'
27
import { ImageResizeError } from './utils/imageResizer.js'
28
import { findToolByName, type ToolUseContext } from './Tool.js'
29
import { asSystemPrompt, type SystemPrompt } from './utils/systemPromptType.js'
30
import type {
31
  AssistantMessage,
32
  AttachmentMessage,
33
  Message,
34
  RequestStartEvent,
35
  StreamEvent,
36
  ToolUseSummaryMessage,
37
  UserMessage,
38
  TombstoneMessage,
39
} from './types/message.js'
40
import { logError } from './utils/log.js'
41
import {
42
  PROMPT_TOO_LONG_ERROR_MESSAGE,
43
  isPromptTooLongMessage,
44
} from './services/api/errors.js'
45
import { logAntError, logForDebugging } from './utils/debug.js'
46
import {
47
  createUserMessage,
48
  createUserInterruptionMessage,
49
  normalizeMessagesForAPI,
50
  createSystemMessage,
51
  createAssistantAPIErrorMessage,
52
  getMessagesAfterCompactBoundary,
53
  createToolUseSummaryMessage,
54
  createMicrocompactBoundaryMessage,
55
  stripSignatureBlocks,
56
} from './utils/messages.js'
57
import { generateToolUseSummary } from './services/toolUseSummary/toolUseSummaryGenerator.js'
58
import { prependUserContext, appendSystemContext } from './utils/api.js'
59
import {
60
  createAttachmentMessage,
Annotations (click the dots)

The query() function in src/query.ts is the heart of Claude Code. It takes a conversation history, calls the Anthropic API with streaming, and yields messages back to the REPL as they arrive.

🔑Key Insight

`query()` is an async generator function — it `yield`s messages one at a time as they stream in, rather than buffering everything and returning at the end. This is what makes the UI feel live.

The pipeline has distinct phases: message normalization → system prompt assembly → streaming API call → tool execution → token budget check → optional compaction. Each phase is independent and testable.

💡Tip

When the API stream fails mid-response, the executor calls `discard()` on the StreamingToolExecutor so any in-flight tool calls are abandoned rather than producing partial results.

KEY TAKEAWAYS
  • The query function is a generator — it yields messages incrementally as they stream
  • Tool calls are handed off to StreamingToolExecutor for parallel execution
  • Token budget is checked after each turn; compaction triggers at ~80% window
  • The pipeline has explicit fallback behavior when streaming fails mid-response
  • Every phase (normalize → system prompt → API → tools → compact) is composable
AI Assistant

Ask anything about Query Engine & Main Loop

Powered by Groq · Enter to send, Shift+Enter for newline