4  The Context Window & Memory Architecture

In casual chatbot usage, the context window is often treated as a simple counter: “how long can my conversation be?” In agentic research workflows, however, the context window is the central operating bottleneck.

Understanding how working memory functions—and how persistent memory is maintained across sessions—is what separates a fragile setup from a robust, production-grade research system.

This chapter explores how the context window operates as working memory, why context pollution degrades agent performance, how the three-tier memory hierarchy protects long-term projects, and why modern architectures default to single-loop progressive disclosure before selectively deploying isolated subagents.


4.1 The Context Window: The Agent’s Working Memory

The context window is the maximum number of tokens an LLM can receive and process in a single inference call. It represents the agent’s total working memory during an active session.

NoteThe Hardware Analogy: The Context Window as “RAM”

In AI systems design, researchers and engineers frequently use computer hardware as an intuitive conceptual metaphor (popularized by Andrej Karpathy’s “LLM OS” model):

  • The LLM is the CPU: The reasoning engine executing instructions and choosing actions.
  • The Context Window is “RAM”: Fast, volatile working memory. Everything loaded into the active prompt is immediately accessible, but it disappears the moment the session ends, resets, or compacts.
  • Disk Files are Secondary Storage (SSD/Hard Drive): Slower to access via tool calls (like reading a file from disk), but persistent and permanent across sessions.

While this analogy is helpful for building intuition about resource scarcity, remember its technical limit: physical computer RAM provides uniform \(O(1)\) random access to any memory address, whereas an LLM’s attention is non-uniform—tokens in the middle of a large prompt are recalled far less reliably than those at the edges (as detailed below).

┌────────────────────────────────────────────────────────────────────────┐
│               THE ACTIVE CONTEXT WINDOW (Working Memory)               │
│                                                                        │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────────┐  │
│  │ System Rules     │  │ Workspace Files  │  │ Conversation & Logs  │  │
│  │ (GEMINI.md, etc) │  │ (Read into chat) │  │ (User & Agent turns) │  │
│  └──────────────────┘  └──────────────────┘  └──────────────────────┘  │
│                                                                        │
└────────────────────────────────────────────────────────────────────────┘

While million-token (and even multi-million-token) context windows have become standard across modern frontier models, having such a massive working memory does not mean you should fill it carelessly.

4.1.1 The “Lost in the Middle” Effect & Attention Dilution

Research has documented a consistent, counter-intuitive quirk in how large language models handle long prompts: their attention follows a U-shaped curve (Liu et al., 2024). Information placed at the very beginning (primacy effect) or the very end (recency effect) of a session is recalled and followed reliably, but instructions and details buried in the middle are frequently missed.

Furthermore, long-context benchmarks like RULER (Hsieh et al., 2024) show that while models can easily find simple keywords in a huge document (“needle-in-a-haystack”), their ability to perform multi-step reasoning, synthesis, or strict rule-following (such as “never modify column X” or “keep equations under 3 lines”) decays noticeably as the conversation grows beyond 50,000–100,000 tokens.

While the exact root causes remain an active area of investigation among AI researchers, the primary intuitive factors include:

  • Attention Dilution: The model has a finite “attention budget.” As the context grows, that budget gets spread across thousands of tokens, lowering the signal-to-noise ratio for any single item in the middle.
  • Natural “Edge” Anchors: System prompts at the very top and recent prompts at the very bottom serve as natural anchors. Models are trained on data where key instructions and questions are at the edges, so the middle is often treated like background text.
  • Positional Fuzziness: Position representations become less sharp over vast token distances, making it harder for the model to link distant concepts.
  • Reading vs. Generating Gap: A model may successfully read a fact in the middle, but fail to route its attention back to that fact when composing its final answer.

4.1.2 Context Pollution: The Silent Performance Killer

The most common reason an agentic workflow starts behaving erratically or ignoring instructions is context pollution. Context pollution occurs when the active conversation becomes cluttered with:

  • Giant terminal error logs (e.g., dumping 500 lines of raw compiler or package installation stack traces).
  • Repeated failed attempts to edit a file without rolling back or clearing the history.
  • Irrelevant files read into the context that have no bearing on the current task.
  • Rambling back-and-forth conversational chatter.

4.1.2.1 The “Telephone Game” (Handoff Context Degradation)

A subtle but pervasive form of context degradation occurs during agent-to-subagent handoffs—a failure mode identified by Anthropic’s Applied AI team as the “Telephone Game” (Anthropic, 2026).

When an orchestrator attempts to break a task apart and hand pieces to specialized subagents, it must summarize and re-prompt the user’s intent:

User Prompt (with critical research nuances)
      │
      ▼
Orchestrator Agent (compresses & paraphrases)
      │
      ├───────────────────────────────┐
      ▼                               ▼
Subagent A ("Telephone" Handoff 1)   Subagent B ("Telephone" Handoff 2)
[Lost: negative constraints]         [Lost: clustering specification]

In academic and empirical workflows, user instructions frequently carry delicate negative constraints (“do not re-estimate Table 2”, “preserve clustering at the firm level”, or “keep equations under 3 lines”). During multi-hop agent handoffs, these critical nuances are often dropped or warped in translation. Dividing work across a swarm of subagents frequently destroys context rather than preserving it.

ImportantContext Hygiene

Treat the context window as a precious resource. A concise, focused 10,000-token session with clear instructions will consistently outperform a bloated 150,000-token session filled with dead-end debugging attempts or lossy multi-agent handoffs.

4.1.3 Context Compaction: Managing Long Sessions

As you work through an extensive research task, your conversation history accumulates thousands of tokens of code edits, terminal outputs, file reads, and intermediate back-and-forth dialogue.

4.1.3.1 What is Context Compaction?

Context compaction is a memory-management process that condenses conversation history into a structured summary to prevent hitting context limits. In Google Antigravity, compaction operates in two complementary modes:

  1. Automatic Compaction (Auto-Compaction): When context utilization approaches capacity (typically ~80%–90% of the active context window), the background harness intercepts the session, summarizes earlier conversational turns with a fast model, prunes bulky tool outputs, and seamlessly continues the thread.
  2. Manual Compaction (/compact): You can proactively run the /compact slash command at any milestone transition (e.g., right after data cleaning is verified, before launching regressions) to clean up working memory on your own terms.
NoteThe Transcript Archive: Compaction is Not Deletion

While compaction summarizes older turns out of the active prompt buffer, Antigravity preserves the full verbatim record on disk in the session directory (.system_generated/logs/transcript.jsonl and transcript_full.jsonl). Full tool outputs, terminal logs, and previous conversational turns remain permanently accessible on disk for auditing and inspection.

4.1.3.2 The Trade-Offs: Pros and Cons

While context compaction is essential for long-running workflows, it comes with important trade-offs:

Advantage (Pros) Limitation (Cons)
Uninterrupted Workflows: Prevents hard “Context Exceeded” errors, allowing complex, multi-hour research tasks to continue in a single thread. Lossy Information Loss: Fine details—such as exact line numbers, raw compiler logs, and nuanced negative constraints—are summarized away from active memory.
Restored Attention Focus: Strips out hundreds of lines of dead-end debugging traces, reducing attention dilution and sharpening the model’s focus. Context Drift & Amnesia: Early conversational decisions or subtle econometric notation conventions agreed upon 20 turns ago can be forgotten.
Lower Cost & Latency: Smaller active prompt sizes reduce per-turn token consumption, preserve KV cache efficiency, and speed up inference times. False Sense of Security: Users may mistakenly assume the agent retains verbatim memory of everything discussed earlier in the session.

4.1.3.3 How Context Compaction Works in Google Antigravity

To ensure critical task details survive compaction, Antigravity coordinates context compaction through a built-in pipeline paired with lifecycle hooks:

NoteWhat is a Lifecycle Hook?

A lifecycle hook is an event-driven script that runs automatically on your computer when a specific platform trigger occurs (such as right before context compaction, or after a tool runs). Unlike LLM instructions—which can be summarized away or forgotten during long sessions—hooks are deterministic code running outside the model’s context window. We explore how to configure custom hooks in depth in Chapter 5.

  1. Token Tracking & Triggering: The harness continuously tracks token consumption. Compaction is triggered automatically at high thresholds or on demand via /compact.
  2. Pre-Compaction State Capture (PreCompact Hook): Before the history is summarized, the pre-compact.py lifecycle hook snapshots the active implementation plan in quality_reports/plans/ and records pending task states.
  3. Summarization & Output Truncation: Earlier dialogue is condensed into a structured summary block. Simultaneously, bulky tool outputs (stack traces, large file reads) are truncated in the active prompt buffer.
  4. Prompt Buffer Replacement: The active prompt history is replaced with the structured summary, preserving system instructions (GEMINI.md) and the most recent interaction turns.
  5. State Restoration (PreInvocation Hook): On the first turn following compaction, post-compact-restore.py detects that compaction occurred and injects an ephemeral message re-anchoring the active plan checklist, ensuring zero task disorientation.
TipThe Context Toolbelt: Compact, Clear, and Rewind
  • /compact: Use when continuing the same task after completing a major milestone to refresh working memory.
  • /clear: Use when switching to a completely unrelated task within the same project to start with a fresh prompt buffer.
  • /rewind / /fork: Use in the CLI or App to roll back a dead-end debugging rabbit hole rather than carrying polluted turns into a compaction cycle.
TipThe Golden Rule of Agentic Memory

Never rely on ephemeral chat history to preserve important project constraints. Use persistent disk files (GEMINI.md for constitutional rules, MEMORY.md for learned preferences, and quality_reports/plans/ for active roadmaps) so critical knowledge survives across compaction cycles and new sessions.


4.2 The Memory Hierarchy in Agentic Workflows

To solve the limitations of ephemeral context windows, a well-designed agentic system uses a three-tier memory architecture. This structure is conceptually modeled on the familiar memory hierarchy in computer hardware:

┌────────────────────────────────────────────────────────────────────────┐
│ LEVEL 1: Ephemeral Working Memory (Hardware Analogy: Volatile RAM)     │
│ Active Context Window • In-flight chat messages • Open file buffers    │
│ Fast, immediate, but wiped clean on session reset or compaction.       │
├────────────────────────────────────────────────────────────────────────┤
│ LEVEL 2: Task Memory (Hardware Analogy: Local SSD / Scratchpad)        │
│ quality_reports/plans/ • Implementation plans • Scratch scripts        │
│ Preserves structured goals and step-by-step checklists across turns.   │
├────────────────────────────────────────────────────────────────────────┤
│ LEVEL 3: Persistent Project Memory (Hardware Analogy: Hard Drive)      │
│ GEMINI.md • MEMORY.md • Session Logs • Git History                     │
│ Endures permanently across days, weeks, and completely fresh sessions. │
└────────────────────────────────────────────────────────────────────────┘
NoteThe Compounding Memory Pattern

This three-tier hierarchy mirrors the knowledge architecture articulated by AI researcher Andrej Karpathy in his LLM Wiki framework: rather than treating the AI as a stateless chatbot that rediscovers facts from scratch on every prompt, the agent incrementally maintains a persistent, compounding codebase of Markdown files on disk. The active chat session is ephemeral working memory; your disk files are the permanent knowledge base.

4.2.1 Level 1: Ephemeral Working Memory (The Chat Window)

  • What it is: The tokens currently inside the active prompt.
  • Lifespan: Single conversation session.
  • Role: Instant communication, quick iterations, scratchpad debugging.

4.2.2 Level 2: Task Memory (quality_reports/plans/)

  • What it is: Markdown plan files written directly to disk.
  • Lifespan: The duration of a specific milestone or task.
  • Role: Protects multi-step tasks against context degradation. If the agent loses focus or auto-compacts, reading the plan file immediately restores full task orientation.

4.2.3 Level 3: Persistent Project Memory (GEMINI.md & MEMORY.md)

  • What it is: Dedicated repository documentation that the agent reads at the start of every session.
  • Lifespan: Permanent (committed to Git).
  • Role: Stores institutional project knowledge:
    • GEMINI.md: The project’s Constitution (folder structure, core commands, notation standards).
    • MEMORY.md: The project’s Corrections Log ([LEARN] tags capturing user preferences and past corrections).
    • quality_reports/session_logs/: A record of why decisions were made.

4.2.4 The Rule Budget & Progressive Disclosure

Because GEMINI.md and always-on rules are prepended to the context on every single turn, they consume working memory and attention.

Empirical testing across LLMs reveals a consistent rule budget:

  • An LLM reliably follows approximately 100 to 150 custom instructions.
  • If your GEMINI.md file balloons into a 600-line manual, the model begins suffering from instruction fatigue and will silently ignore rules.
  • Solution: Keep GEMINI.md under ~120 lines for top-level principles, and move specialized domain guidelines into path-scoped rules (which only load when editing matching files like .R or .tex).

4.2.4.1 The Architecture of Progressive Disclosure

In production agent design, modern systems manage the rule budget through Progressive Disclosure—an architectural principle championed by Anthropic in their engineering of Claude Code (Anthropic, 2026):

Instead of front-loading every rule, workflow instruction, and tool definition into the system prompt upfront, the agent is initialized with a minimal surface and discovers or loads detailed procedures just-in-time as task requirements unfold.

Anthropic demonstrated that adopting progressive disclosure allowed them to reduce their core system prompt by over 80%, recovering massive amounts of attention and context budget.

┌────────────────────────────────────────────────────────────────────────┐
│                        PROGRESSIVE DISCLOSURE                          │
│                                                                        │
│  TIER 1: Lean Constitutional Core (~100 lines)                         │
│  GEMINI.md / System Rules (Always in context; high-level standards)   │
│                                  │                                     │
│                                  ▼ on-demand invocation                │
│  TIER 2: Modular Skills (SKILL.md)                                     │
│  Loaded into active context ONLY when specific task triggers           │
│                                  │                                     │
│                                  ▼ dynamic parameter resolution        │
│  TIER 3: Deferred Tool Search & Loading                                │
│  Lightweight tool manifests; full tool schemas fetched on call         │
└────────────────────────────────────────────────────────────────────────┘

In Google Antigravity, this three-tier disclosure is built directly into the harness architecture to keep the model’s active working memory uncluttered:

Tier Layer Where It Lives When Loaded Research Purpose & Examples
Tier 1 Constitutional Core GEMINI.md & path rules Always On
(Every prompt turn)
Non-negotiable project boundaries, coding conventions, and repository layout (e.g., “XeLaTeX only”, “verify numbers against logs”, “keep files clean”).
Tier 2 Modular Skills .agents/skills/*/SKILL.md Just-in-Time
(Only when invoked)
Deep, multi-page procedural manuals and estimation recipes (e.g., DiD protocols in r-econometrics, 10-K parsing in sec-edgar, or BibTeX citation standards).
Tier 3 Deferred Tools MCP & built-in tool schemas On Demand
(When tool is called)
Detailed parameter definitions and API contracts. The model sees compact tool names first; heavy JSON schemas are retrieved only when the tool is executed.
TipWhy Progressive Disclosure Matters for Researchers

If you bundled all econometric estimation guidelines, LaTeX formatting manuals, and 40+ tool schemas directly into GEMINI.md, your agent would start every session with 25,000+ tokens of static overhead. Progressive disclosure keeps the initial prompt under ~1,500 tokens, leaving over 95% of the model’s attention budget free for your actual data, code, and econometric derivations.


4.3 Context Caching and API Economics

In large academic projects—where an agent session might carry hundreds of pages of LaTeX code, empirical regression scripts, and extensive literature bases—re-processing that entire context on every turn would quickly become slow and expensive.

4.3.1 How Context Caching Works

Both Google Gemini (Antigravity) and Anthropic (Claude) utilize Prompt / Context Caching to eliminate this bottleneck:

  • Neural Activation Reuse: When static context (such as system instructions, GEMINI.md, and loaded background files) remains unchanged between turns, the API server caches the computed Key-Value (KV) activations in memory.
  • Massive Cost and Latency Reductions: Subsequent turns read from the cache at a 75%–90% cost discount and with near-instant processing latency (slashing response times from 15+ seconds down to 2–3 seconds).

4.3.2 The Two Golden Rules of Cache Efficiency

To maximize context caching in your research workflows, keep two principles in mind:

  1. Maintain Session Momentum (The 1-Hour Cache Window): Prompt caches are temporary. On modern provider backends and subscription tools (including Google Antigravity and Claude Code), conversational prompt caches typically use a 1-hour Time-To-Live (TTL).

    Each continuous turn keeps the cache hot. A 1-hour window gives you ample breathing room to step away, review regression logs, or read a paper draft without the cache going cold. However, if you leave a session idle for several hours or overnight, the server evicts the cache, meaning your next prompt will incur a one-time cold-start latency pass to re-warm the context from scratch.

  2. Preserve Prefix Stability: Context caching matches the exact token prefix starting from the very first token in the prompt. If you edit base configuration files like GEMINI.md or modify top-level system rules midway through a session, the prefix changes, invalidating the cached activations. Keeping base rules stable and placing task-specific details in plan or script files preserves unbroken cache hits.

4.3.3 Single-Loop Cache Warmth vs. Multi-Agent Cold Starts

Context caching introduces a profound, counter-intuitive insight into agentic performance: a single, continuous agent loop is often twice as fast and substantially cheaper than a sprawling multi-agent system.

  • Single Loop (Hot Cache): A single agent iterating on a task repeatedly queries the model with an incremental turn appended to an already-cached prompt prefix. The base system rules, workspace index, and historical steps are retrieved from the provider’s KV cache at a 75%–90% cost discount and with instantaneous 2–3 second response latencies.
  • Multi-Agent Swarm (Cold Starts & Cache Misses): When an orchestrator spins up multiple distinct subagents, each subagent initializes its own brand-new context window. Each separate context incurs a cold-cache penalty, pays full price for initial prompt tokens, and adds cumulative round-trip latency. In real-world enterprise deployments, collapsing redundant subagent hierarchies into a single loop running with progressive disclosure cut execution times by ~50% while noticeably decreasing overall token spend (Anthropic, 2026).

4.4 Looking Ahead: Single-Loop Default vs. When Subagents “Earn Their Keep”

When developers and academics first discover agentic workflows, there is an instinctive urge to over-engineer: creating an “intro agent”, a “literature agent”, a “regression agent”, a “proofreading agent”, and an “orchestrator agent” to manage them all.

This mirrors what software engineers call Conway’s Law (Anthropic, 2026): teams divide multi-agent systems to reflect human organizational charts or mental silos rather than the optimal mathematical and computational architecture.

4.4.1 The Single-Loop Default

For the vast majority of research tasks, your default architecture should be a single, open-ended agent loop that maintains and revises its own plan.

A single agent loop equipped with the following capabilities consistently outperforms a swarm of subagents:

  • Disk-based task memory: An active implementation plan written to disk (quality_reports/plans/task_plan.md).
  • Progressive disclosure: On-demand skills loaded dynamically only when needed.
  • Direct environment tools: Continuous read/write workspace and terminal execution tools.

By keeping execution in a single thread, the agent maintains unbroken ownership of the user’s intent, avoids handoff degradation (“the Telephone Game”), preserves hot context caches, and can dynamically adjust its hypothesis when an econometric script fails without having to coordinate across three intermediate agents.

4.4.2 The Three-Part Subagent Justification Test

Subagents should not be eliminated entirely, but they must earn their keep. In production academic workflows, subagents are justified only when a task satisfies one of three rigorous architectural criteria:

                     IS A SUBAGENT JUSTIFIED?
                                │
        ┌───────────────────────┼───────────────────────┐
        ▼                       ▼                       ▼
 1. Disposable Context   2. True Parallel        3. Fresh-Context
    "Trash Can"             Fan-Out                 Verification
  (Isolate heavy inputs   (Concurrent searches   (Unbiased adversarial
   like 300-page SEC/PDFs  with zero shared       audit without prior
   protects main context)  state dependencies)     confirmation bias)
        │                       │                       │
        └───────────────────────┼───────────────────────┘
                                │ YES
                                ▼
                       DISPATCH SUBAGENT
  1. The Disposable Context Sink (“Trash Can” Pattern):
    • Problem: Ingesting massive raw data—such as parsing a 300-page SEC 10-K filing, reading 500 lines of raw compiler error logs, or scraping 30 web pages—permanently bloats and pollutes the primary agent’s working memory.
    • Solution: Dispatch an isolated subagent as a disposable context. The subagent reads the raw text, extracts the specific numbers or findings, writes the distilled synthesis back to disk or chat, and its bloated context window is promptly discarded. The main loop remains lean and pristine.
  2. Genuinely Parallel Fan-Out:
    • Problem: Some research steps involve searching across multiple independent sources that have zero dependencies on each other. Running them sequentially in a single loop wastes significant wall-clock time.
    • Solution: Dispatch multiple lightweight subagents concurrently—for example, searching OpenAlex, querying the FRED database, and checking SSRN working papers simultaneously—and synthesize their findings once all return.
  3. Fresh-Context / Adversarial Verification (The Independence Principle):
    • Problem: An agent that has spent 40 turns deriving a proof or writing an empirical analysis script suffers from self-confirmation bias. If asked to review its own work within the same session, it will instinctively defend its earlier logic, skim past subtle errors, and rubber-stamp its conclusions.
    • Solution: Spawn an independent reviewer subagent with a 100% clean, blank-slate context window (such as domain-referee or claim-verifier). The reviewer receives only the output file and the verification rubric. Free from the conversational history and emotional baggage of how the result was derived, the subagent performs a truly objective, adversarial audit.

In Chapter 5 (The Building Blocks), we will explore how Google Antigravity implements these specialized subagents alongside modular skills and lifecycle hooks.


4.5 Summary: Context & Memory Checklist for Scholars

  1. The Context Window is Ephemeral Working Memory: Treat it like an operating system’s RAM—fast and immediate, but volatile and scarce. Avoid dumping raw terminal traces, unpruned file dumps, or rambling chatter into the conversation.
  2. Default to a Single Open-Ended Loop: Keep task ownership in one continuous loop that maintains its own plan on disk (quality_reports/plans/).
  3. Use Progressive Disclosure: Keep the core Constitution (GEMINI.md) lean (~120 lines). Load procedural knowledge (SKILL.md) and tool schemas dynamically on demand rather than front-loading everything.
  4. Beware the Telephone Game: Subagent handoffs degrade nuance and lose negative constraints. Keep tightly coupled steps in the main loop.
  5. Enforce the Subagent Justification Test: Only spawn subagents for disposable context sinks (heavy file ingestion), true parallel fan-out, or fresh-context adversarial verification.
  6. Use Disk Files as Persistent Task Memory: Save multi-step roadmaps to disk (analogous to writing from volatile RAM to permanent SSD storage) so goals survive session compaction and restarts.
  7. Capture KV Cache Hits: Keep sessions active within the ~1-hour TTL window to benefit from the 75%–90% cost discount and sub-3-second latencies.
TipEmpirical Architecture: “Evals Over Vibes”

When deciding whether to split a research workflow across subagents or fold it into a single loop, resist designing by intuition. Follow the engineering discipline in Anthropic’s single-agent-loop-kit: establish a minimal benchmark of 10 representative golden test cases with verified outputs. Measure task success rate, token cost, and wall-clock latency across multiple runs before committing to architectural complexity.