vectorize-io/hindsight

What Hindsight is and the problem it targets

Hindsight is an agent memory system built by Vectorize and released under the MIT license. According to the opening of its README, the goal is to create agents that get smarter over time, meaning agents that learn rather than just remember. The authors stress that distinction in the very first lines.

The README argues that most agent memory systems focus on recalling conversation history. Hindsight aims to go further: the authors say it removes the shortcomings of alternatives such as RAG and knowledge graphs and delivers state-of-the-art performance on long-term memory tasks. That is the authors' own claim, so you should test it on your own data before relying on it.

On popularity, the repository metadata shows 53775 stars, with 2072 gained over the last seven days, and a most recent push on October 25, 2026. These numbers only show community interest and do not prove quality, but they do suggest the project is actively maintained.

Performance and accuracy as reported by the authors

The README has a dedicated section on memory performance and accuracy. The authors state that Hindsight is the most accurate agent memory system ever tested according to benchmark performance, and that it reached state-of-the-art results on LongMemEval, a benchmark widely used to assess memory systems across a range of conversational AI scenarios. The comparison point given in the README is January 2026.

The README says the benchmark data for Hindsight was independently reproduced by research collaborators at the Virginia Tech Sanghani Center for Artificial Intelligence and Data Analytics and at The Washington Post, while the scores of other systems are self-reported by their vendors. This is still information written by the authors, and the text we have does not include specific score values, so this article quotes no scores.

The README also says live, continuously updated results, including per-model accuracy, latency and cost, are published on a separate benchmarks site run by the project. The authors add that Hindsight is used in production at Fortune 500 enterprises and by a growing number of AI startups. The README names no customers and gives no scale, so treat that as background rather than evidence.

Memory types: world facts, experiences, observations and mental models

Instead of relying only on vector search or a knowledge graph, Hindsight says it uses biomimetic data structures to organize memories in a way closer to human memory. The README splits memory into four kinds: world facts, such as the stove gets hot; the agent's own experiences, such as I touched the stove and it hurt; observations; and mental models.

Observations are consolidated, evidence-backed beliefs formed from many memories. Mental models are the learned understanding of the agent's world, synthesized from observations and facts. This layering lets an agent keep not only raw facts but also the conclusions drawn from them.

When memories are added, the system pushes them into either the world-facts pathway or the experiences pathway, then represents them as a combination of entities, relationships and time series, together with sparse and dense vector representations to aid later recall. All memories live in banks, a concept covered below.

The three core operations: retain, recall and reflect

Retain pushes new memories into Hindsight. The README explains that behind the scenes, retain uses a language model to extract key facts, temporal data, entities and relationships, then normalizes them into canonical entities, time series and search indexes with metadata. These representations create the pathways that make recall and reflect accurate.

Recall retrieves memories, and they can come from any memory type. The README says recall runs 4 retrieval strategies in parallel: semantic search by vector similarity, keyword search with BM25 exact matching, graph search over entity, temporal and causal links, and temporal filtering by time range. The results are merged, ordered with reciprocal rank fusion and a cross-encoder reranking model, then trimmed to fit the token limit.

Reflect performs a more thorough analysis of existing memories, letting the agent form new connections or answer a question that needs deep thinking rather than lookup. The README gives three examples: an AI project manager reflecting on which risks need mitigation, a sales agent working out why some outreach messages got responses and others did not, and a support agent finding customer questions that current product documentation does not answer.

Observations, mental models and knowledge pages

The README stresses that retained facts do not stay a flat pile. In the background, Hindsight consolidates related facts into observations, which are deduplicated beliefs the bank has built up over time. Each observation keeps its supporting evidence with exact quotes and a proof count, and it is refined rather than overwritten when new evidence arrives. New information can therefore strengthen, weaken or extend an existing belief instead of silently replacing it.

A mental model is a standing answer to a question about a bank, for example what this user's preferences are. You define the question once, and Hindsight writes the answer, stores it and rewrites it in the background as the bank learns more. The README says reading one is just a database read, with no retrieval and no language model call, so an agent can boot with a page of settled knowledge instead of rediscovering it every session.

Knowledge pages are mental models with the mechanics hidden: living documents a bank writes about itself, organized in folders like a wiki, searchable, and projectable onto disk as ordinary markdown. You supply a name and a question, and every other decision is a default you can override.

Banks, multilingual handling and Memory Defense

A bank is an isolated memory store, effectively one brain for one user, agent or project. The README says isolation is strict, with no cross-bank leakage. Banks carry background context and disposition traits such as skepticism, literalism and empathy, which shape how reflect reasons over their memories, and they can be created from declarative bank templates.

Hindsight is described as multilingual by default. The input language is detected and preserved end to end, facts stay in their original language and entities keep their native script. The README gives the example of a Chinese personal name that stays in Chinese characters rather than being transliterated into Latin letters.

Memory Defense is an opt-in policy configured per bank. On every retain, the system scans for secrets and personal data against 45 patterns, then either redacts the match or blocks the item before it reaches storage. For agents that handle real user data this is a layer worth considering, but because it is opt-in you have to turn it on yourself.

Installation and first run

The README offers several routes. Docker is the recommended one, exposing an API and a web UI. There is also Docker with an external PostgreSQL, bare metal installation with pip, Kubernetes deployment with Helm, and a managed service called Hindsight Cloud, where you point a client at the service with an API key and skip running anything yourself. The README says Cloud is billed by usage, with free credits to start, no fixed monthly or per-seat fee, and a 99.9 percent uptime SLA.

Hindsight uses a language model for steps such as retain, and the README says it works with more than 25 providers, selected through an environment variable. The list covers hosted services, fully local options such as ollama, lmstudio and llamacpp, any OpenAI-compatible endpoint, and gateways such as litellm. Some existing subscriptions, including ChatGPT Plus or Pro, Claude Pro or Max, Cursor and GitHub Copilot, also work without an API key.

For clients, the README lists Python, Node.js or TypeScript, Go, a CLI and a REST API. The supported platform table shows Linux, macOS on Apple Silicon and Windows all working with Docker, pip and the embedded pg0 database. On Intel Macs the pip route carries a warning and the README points to the hindsight-all-slim package. There is also an embedded mode for Python that needs no separate server, plus a Node.js equivalent and a daemon CLI.

Using Hindsight in an agent workflow

The fastest way to add memory to an existing agent is the LLM Wrapper. You swap your language model client for a wrapped one, and memories are then stored and retrieved automatically on every call, with no other code changes. The README says a matching wrapper exists for the Anthropic SDK, and every setting, such as the bank, recall budget, fact types, or using reflect instead of recall, can be overridden per call. LiteLLM sits underneath, so the same integration covers more than 100 models. If you need explicit control over when memories are stored and recalled, the README advises using the SDKs or the REST API directly.

The README claims more than 60 integrations, most needing no code changes. Coding agents include Claude Code, Codex, Cursor, GitHub Copilot, opencode, Cline, Aider, Zed, Continue, Roo Code and OpenHands. Agent frameworks include LangGraph and LangChain, LlamaIndex, CrewAI, Pydantic AI, OpenAI Agents SDK, Google ADK, Agno, Strands, AutoGen, Microsoft Agent Framework, Vercel AI SDK and Haystack. There are also n8n, Zapier, Dify and Flowise, plus ChatGPT, Perplexity, Obsidian, Pipecat and Vapi.

For command-line coding agents, the README describes a package that provides long-term project memory: a per-repo bank built automatically from git history and past sessions, injected into the agent as it starts working, with knowledge pages covering architecture, conventions and in-flight work. Ingestion is automatic and there is no setup command. In addition, every server ships a built-in Model Context Protocol endpoint per bank, enabled by default, so any MCP client can use retain, recall and reflect as tools.

Running it in production

The Running in Production section of the README summarizes the operational pieces. Storage uses PostgreSQL with pgvector, or Oracle AI Database 23ai with full feature parity. Configuration is hierarchical, from global environment variables down to each tenant and then each bank. Monitoring relies on Prometheus metrics and dashboards for language model calls, tokens and latency.

For operations there is an admin CLI for migrations, bank repair and stuck operations. Webhooks cover the retain, consolidation and refresh lifecycle events. Extension points exist for tenants, authentication and storage. For teams that do not want to carry this burden, the authors point to Hindsight Cloud. The README does not go deep on architecture or large-scale performance, so questions about capacity, language model cost and real latency need to be measured on your own system.

Limits and when to use it

The README names one notable limit itself: Hindsight fits best for agents that need a blend of features, such as AI employees that handle open-ended tasks, change behavior based on user feedback and learn complex tasks. For simple AI workflows built with n8n and similar tools, the authors say Hindsight may be overkill. The simplest example use case is personalizing chatbots per user, using metadata at retain time to isolate memories and then filtering at retrieval time.

Some things the README does not state clearly and you should verify yourself: concrete benchmark scores, the language model cost of retain and reflect on large data, and the quality of automatically generated observations in your own domain. The README also marks Hindsight as a trademark and ties the project to Vectorize, so if you use the Cloud service you depend on a commercial provider.

In short, if you are building an agent that must remember for a long time, learn from feedback and keep knowledge across sessions, Hindsight is worth trying, especially if you already use a framework or coding agent from its integration list. If your agent only runs short, stateless tasks, a memory layer like this may just add complexity. A safe path is to start with Docker or the embedded mode, try the LLM Wrapper on a small agent, and measure recall quality yourself before going to production.