Agents & Coding

Local Indexing to reduce tool calling and token usage

Pensieve claims local MCP indexing cuts agent lookups to 2.0 tool calls versus 3.4 for grep, but the benchmark is vendor-run and the discussion questions differentiation in a crowded field.

80 points22 comments

Pensieve is a local macOS search and indexing tool for files, git repositories, and coding-agent sessions. It exposes results to agents over MCP so they can retrieve relevant passages with file and line instead of repeatedly grepping and reading whole files.

The project reports a vendor-run benchmark on 72 held-out tasks across four repos, claiming fewer tool calls, lower cost, and faster lookups, with local indexing and optional AI.

Key takeaways

  • Pensieve indexes local files, folders, git repos including uncommitted changes and worktrees, and Claude Code, Codex, and Qwen Code sessions; search runs on-device.
  • Agents query it over MCP via stdio or HTTP; pensieve integrate configures Claude Code, Codex, Cursor, Gemini CLI, Windsurf, Claude Desktop, and VS Code, adding a 'search before you grep' instruction block.
  • The reported benchmark used Claude Sonnet headless in Claude Code on 72 held-out tasks across four repos (504 to 34,847 files), run twice; Pensieve found the right file in 131/144 runs versus 123/144 with grep alone.
  • The project claims 43% fewer tool calls per lookup (2.0 vs 3.4), 18% lower cost ($0.041 vs $0.050), 21% faster lookups (8.6s vs 10.9s), and 16% fewer tokens; benchmark scripts are open source.
  • Privacy claims: indexing and search stay local, sources can be toggled off, secrets such as .env files and password-manager exports are skipped, and AI insights are optional.
  • Requirements: Apple Silicon, macOS 14 or later; Linux and Intel Macs get backend and browser visualizer, with Intel search panel requiring Xcode command-line tools. MIT licensed.

What people think

Agent code search and context retrieval

Pensieve is pitched for coding agents that need to find code, docs, or past sessions without repeated grep-and-read loops.

Crowded agent-search tooling

A commenter says many similar tools launch monthly and Cursor previously had comparable functionality, though session referencing stands out.

Token and lookup cost

The creator argues per-lookup savings accumulate across hundreds of daily agent searches.