MEM0：一个新的以内存为中心的LLMS，可在会话中保留信息

大型语言模型可以产生流利的响应，模仿音调，甚至遵循复杂的说明；但是，他们很难在多个会议上保留信息。

Large language models (LLMs) are revolutionizing natural language processing (NLP) with their ability to generate fluent responses, emulate tone, and follow complex instructions. However, these models still struggle with a critical limitation: they have difficulty retaining information across multiple sessions.

大型语言模型（LLM）正在彻底改变自然语言处理（NLP），其能力产生流利的响应，效仿音调并遵循复杂的说明。但是，这些模型仍然在关键的局限性上挣扎：它们很难在多个会议上保留信息。

This limitation becomes increasingly pressing as LLMs are integrated into applications that require long-term engagement with users. From personal assistance and health management to tutoring and more specialized tasks, the seamless flow of conversation is paramount. In real-life conversations, people recall preferences, infer behaviors, and construct mental maps over time. A person who mentioned their dietary restrictions last week expects those to be taken into account the next time food is discussed. Similarly, a user who described their hometown yesterday anticipates the LLM to recognize it and use it in later greetings. Without mechanisms to store and retrieve such details across conversations, AI agents fail to offer the consistency and reliability expected from them, ultimately undermining user trust.

随着LLM集成到需要与用户长期参与的应用程序中，该限制变得越来越紧迫。从个人帮助和健康管理到辅导和更专业的任务，无缝的对话流程至关重要。在现实生活中，人们会回想起偏好，推断行为并随着时间的推移构建心理图。上周提到饮食限制的人期望将这些人考虑到下一次食物。同样，昨天描述其家乡的用户预计LLM会识别出来并在以后的问候中使用它。如果没有在对话中存储和检索此类详细信息的机制，AI代理就无法提供预期的一致性和可靠性，最终破坏了用户信任。

The central challenge with today’s LLMs lies in their inability to persist relevant information beyond the boundaries of a conversation’s context window. These models rely on a limited capacity for tokens, which are units of language used by the model, with some models having a capacity of as high as 128K or 200K tokens. However, when long interactions span days or weeks, even these expanded windows become insufficient. More critically, the quality of attention—the model’s ability to focus on and process specific tokens—degrades over more distant tokens, rendering it harder for the model to locate or utilize earlier context effectively. For instance, a user may personally introduce themselves, switch to a completely different topic like astronomy, and only much later return to the original subject to ask for the personally mentioned fact. Without a robust memory system, the AI will likely ignore the previously mentioned details and instead answer based on the last 10 messages, which in this case would be about astronomy, leading to an incorrect reply. This creates friction and inconvenience, especially in scenarios where continuity and accuracy are crucial. The issue is not just about the model forgetting information, but also about it potentially retrieving the wrong information from irrelevant parts of the conversation history due to token overflow and thematic drift.

当今LLMS的核心挑战在于他们无法在对话上下文窗口的边界之外持续相关信息。这些模型依赖于有限的代币能力，该代币是模型使用的语言单位，有些模型的容量高达128K或200K令牌。但是，当长时间的互动跨越几天或几周时，即使这些扩展的窗户也变得不足。更重要的是，关注质量（模型专注于和处理特定令牌的能力）在更遥远的令牌上，使模型更难找到或使用早期的上下文。例如，用户可以亲自自我介绍，切换到一个完全不同的主题，例如天文学，只有很久以后返回原始主题，以要求个人提到的事实。如果没有强大的内存系统，AI可能会忽略上述详细信息，而是根据最近10条消息的答复，在这种情况下，这将是关于天文学的，导致答复不正确。这会造成摩擦和不便，尤其是在连续性和准确性至关重要的情况下。问题不仅涉及模型忘记信息，还涉及到它可能从对话历史记录中无关的部分中检索错误的信息，这是由于令牌溢出和主题漂移而导致的。

Several attempts have been made to address this memory gap. Some systems, like those from Google AI and Stanford, rely on retrieval-augmented generation (RAG) techniques. These systems use a separate component to search for and retrieve relevant text chunks from a large knowledge base or prior conversations using similarity searches. Another category of systems employs full-context approaches, where the entire conversation history is simply re-fed into the model at the beginning of each turn. Finally, there are proprietary memory solutions like OpenAI’s Memory API and open-source alternatives like PEGASO, which try to store past exchanges in specialized vector databases or structured formats. However, these methods often lead to inefficiencies. For instance, RAG systems can retrieve excessive irrelevant information, while full-context approaches increase latency and token costs. Proprietary and open-source solutions may struggle to consolidate updates to existing memories in a meaningful way, and they lack effective mechanisms to detect conflicting data or prioritize newer updates. This fragmentation of memories hinders the models’ ability to reason reliably over time.

已经尝试了几次解决此记忆差距的尝试。某些系统，例如Google AI和Stanford的系统，依赖于检索功能（RAG）技术。这些系统使用单独的组件来搜索和检索使用相似性搜索的大型知识库或事先对话中的相关文本块。另一类系统采用了全文的方法，在每个回合开始时，整个对话历史记录都简单地将其重新介绍到模型中。最后，有专有内存解决方案，例如OpenAI的内存API和Pegaso等开源替代方案，它们试图将过去的交换存储在专用矢量数据库或结构化格式中。但是，这些方法通常导致效率低下。例如，抹布系统可以检索过多的无关信息，而全文方法的方法会增加延迟和代币成本。专有和开源解决方案可能难以以有意义的方式巩固对现有记忆的更新，并且缺乏有效的机制来检测矛盾的数据或优先考虑更新的更新。记忆的这种分裂阻碍了模型随着时间的推移可靠推理的能力。

To address these limitations, a research team from Mem0.ai developed a novel memory-focused system called Mem0. This architecture introduces a more dynamic mechanism to extract, consolidate, and retrieve information from conversations as they unfold. The design of Mem0 enables the system to systematically identify useful facts from ongoing interactions, assess their relevance and uniqueness, and integrate them into a persistent memory store that can be consulted in future sessions. In essence, Mem0 is capable of "listening" to conversations, extracting key facts, and updating a central memory with these facts. The researchers also proposed a graph-enhanced version of the system, denoted as Mem0g, which builds upon the base system by structuring information in relational formats, connecting facts through entities and their properties.

为了解决这些局限性，MEM0.AI的研究团队开发了一种新型以内存为中心的系统，称为MEM0。该体系结构引入了一种更具动态的机制，可以在展开时从对话中提取，合并和检索信息。 MEM0的设计使系统能够从持续的互动中系统地确定有用的事实，评估其相关性和独特性，并将其集成到持续的内存商店中，并在将来会议中咨询。从本质上讲，MEM0能够“聆听”对话，提取关键事实并使用这些事实更新中央记忆。研究人员还提出了该系统的图形增强版本，称为MEM0G，该版本通过以关系格式构造信息，通过实体及其属性结合事实来建立基础系统。

These models were tested using the LOCOMO benchmark, a standard framework for evaluating conversational memory systems. They compared six categories of memory-enabled systems: memory-augmented agents, RAG methods with varying configurations, full-context approaches, and both open-source and proprietary tools. The goal was to assess these systems' ability to process a wide range of question types, from single-hop factual lookups to multi-hop and open-domain queries.

使用机车基准测试了这些模型，这是评估对话记忆系统的标准框架。他们比较了六种支持内存的系统：内存仪器的代理，带有不同配置的抹布方法，全文使用方法以及开源和专有工具。目的是评估这些系统处理各种问题类型的能力，从单跳事实查找到多跳和开放域查询。

The core of the Mem0 system involves two operational stages. In the first phase, the model processes pairs of messages, typically a user’s question and the assistant’s response, along with summaries of recent conversations. A combination of a global conversation summary over the last hour and the last 10 messages serves as the input for a large language model (LLM) that extracts salient facts. For instance, if the user asks "What is the capital of France?" and the assistant responds with "The capital of France is Paris," the fact extractor would identify "capital_of(France,

MEM0系统的核心涉及两个操作阶段。在第一阶段，模型处理了一对消息，通常是用户的问题和助手的回答，以及最近对话的摘要。最后一个小时的全球对话摘要和最后10条消息的结合是提取显着事实的大语言模型（LLM）的输入。例如，如果用户问“法国的首都是什么？”助理以“法国的首都是巴黎”的回应，事实提取器将确定“ Capital_of（法国，，