I recently wrote about how having proper long-term memory/knowledge about the user/team is such a critical step to make AI assistants smarter. Without it, no matter how advanced LLM gets, for your repeated use cases, it's always day one and you have to keep spending time on the "prompt dance" – that repetitive process of crafting and re-crafting prompts to get what you want in one shot.
We have to move past this "day one" problem.
I also wrote about what kind of information should be injected, depending on the task. It's similar to model routing but more like memory routing, because otherwise retrieval is inefficient and less accurate.
That's why I was excited to come across MIRIX, that aligns with these ideas.
The Problem with Current Memory Systems
Many of today's memory systems, including Letta, Mem0, and ChatGPT's memory system, store all historical data in a single flat store. This means they do not route information into specialized memory types. The result is inefficient and less accurate retrieval.
Other systems like Zep and Cognee take a different approach, using knowledge graphs to represent structured relationships between entities. However, these struggle to model sequential events, emotional states, full-length documents, or multi-modal inputs like images.
MIRIX's Modular Solution
MIRIX addresses these limitations by using a modular architecture with six distinct components: Core Memory, Episodic Memory, Semantic Memory, Procedural Memory, Resource Memory, and Knowledge Vault.
Here's a breakdown of each type and their purpose:
• Core Memory: Stores high-priority, persistent information that should always be visible to the agent, such as the agent's persona and enduring facts about the user (e.g., name, preferences). It's designed to maintain compactness by triggering a rewrite process if memory size exceeds 90% capacity.
• Episodic Memory: Captures time-stamped events and interactions, functioning as a structured log or calendar to enable the agent to reason about user routines, recency, and context-aware follow-ups. Entries include event type, summary, details, actor, and timestamp.
• Semantic Memory: Maintains abstract knowledge and factual information independent of specific times or events, serving as a knowledge base for general concepts, entities, and relationships, whether about the world or the user's social graph. Entries include name, summary, details, and source, and are intended to persist unless conceptually overwritten.
• Procedural Memory: Stores structured, goal-directed processes like how-to guides, workflows, and interactive scripts. This knowledge assists the user with complex tasks and includes entry type, goal description, and a list of steps.
• Resource Memory: Handles full or partial documents, transcripts, or multi-modal files that the user is actively engaged with but do not fit into other categories. It enables context continuity in long-running tasks and stores details like title, summary, resource type, and content.
• Knowledge Vault: Acts as a secure repository for verbatim and sensitive information such as credentials, addresses, contact information, and API keys. These entries are crucial for authenticated tasks and are protected via access control to prevent misuse or leakage.
This modular design aims to a) address the lack of compositional memory structure for efficient and accurate retrieval, b) multi-modal support to handle non-text input efficiently, c) enhance scalability and abstraction (e.g., on the ScreenshotVQA benchmark, MIRIX reduced storage requirements by 99.9% compared to RAG baselines and 93.3% compared to long-context baselines), and d) enable effective routing and retrieval.
Recent works have proposed finer-grained memory architectures. Some of these emphasize the importance of episodic memory, others incorporate both episodic and semantic memory, and some identify procedural memory as crucial. While these individual types have been recognized, the sources state that "existing methods stop at identifying individual memory types, and they are not formed into a comprehensive memory system" like MIRIX.
The Local Implementation Opportunity
This becomes especially powerful when applied to teams like engineering. Imagine having this entire system dedicated to your specific job context. When focused on team productivity, you can accumulate your team's collective knowledge, creating an assistant that's no longer stuck starting from day one.
The implementation could look like a package in your monorepo. When you run it, it opens a browser interface similar to Codex. It understands your codebase because it lives alongside it. All memory can live locally in Postgres, or sync to the cloud to stay in sync with your entire team.
For engineering teams, this offers several concrete benefits:
• Institutional knowledge preservation: New team members can quickly understand project history, decision rationale, and coding patterns that would otherwise take months to absorb
• Consistent code quality: The system learns your team's specific coding standards, architectural decisions, and best practices, helping maintain consistency across contributors
• Reduced onboarding time: Instead of spending weeks learning project-specific workflows and conventions, new developers can query the system for instant context
• Context switching efficiency: When developers switch between projects or return from time off, the system can quickly restore relevant context about what they were working on and why
• Debugging acceleration: The system remembers previous similar issues, solutions that worked, and debugging approaches that were effective for your specific codebase
That memory module could become one of your team's most valuable internal assets – a living repository of not just what was built, but how and why it was built that way.