正在打开每日关注
正在打开每日关注
解析朴素 RAG 的过时与耦合问题，以及持续同步、统一检索和 Agent 循环调用的演进路径。
Google and Anthropic agree on one thing about retrieval. They both moved it out of the app and turned it into a standalone service that agents invoke. - Anthropic's MCP exposes retrieval as a tool that an agent invokes. - Google ships it similarly, with its RAG Engine sitting under the Gemini agent platform next to MCP servers. They did this because the old naive RAG setup was built as a one-time pipeline inside a single app, and it caused problems in two places: - The first is stale embeddings. They reflect the source at indexing time, so the index drifts from the source when a doc or a row changes, and stays wrong until the pipeline is rerun. - The second is coupled infrastructure. The retrieval logic sits inside one app, so another app duplicates the connectors, chunking, and embedding instead of reusing them. Both failure modes are resolved by separating ingestion from query and running ingestion as a standing layer. Ingestion runs continuously with content-hash sync, so only changed data is re-embedded, and the layer sits behind one API, so a chatbot and an agent hit the same index. The agent then calls that layer as a tool inside a loop. It reasons, runs a search, reads the result, and issues a refined query, instead of retrieving once at the start. The diagram below depicts all three stages, covering naive RAG, the standing layer, and the agent consuming it. This helps, but it didn't solve all problems yet because across all three stages, the unit of retrieval is the same, i.e., a chunk of text. And inherently, a chunk has no idea of a semantic boundary, version, or source info. The splitter cuts documents mostly on token count, so a single chunk can end mid-table or mid-argument. Because of that, it can pull the top half of a table without the rows, or a conclusion without the reasoning that supports it. Moving to a structured unit, embedding a question with its validated answer, addresses this. I wrote wrote a full breakdown on solving this. It replaces raw chunks with structured blocks that carry their own source and version. The approach reduces corpus size 40x and improves vector relevance 2.3x. Read it below.
是什么 这条推文把 RAG 的演进拆成三层：朴素 RAG 在单一应用中离线摄取、切块、嵌入并一次检索；Retrieval Layer 将连接器、清洗/切块、嵌入和向量库做成持续运行、可共享的服务；Agentic Layer 则让 Agent 在运行时把检索当工具，按“推理—搜索—阅读—改写查询—行动”循环调用。
核心论点 作者认为，一次性管线会造成来源变化后的陈旧嵌入，也会让多个应用重复建设连接器和检索逻辑。内容哈希同步只重算变化数据，统一 REST/MCP 接口则让聊天机器人与 Agent 复用同一索引；查询扩展、混合搜索、重排也被放进统一搜索接口。
关键洞察 真正的瓶颈不只在部署位置，还在检索单元：按 token 切出的 chunk 缺少语义边界、来源和版本，可能截断表格或论证链。推文进一步主张用携带来源/版本的结构化块，并将经过验证的问题—答案共同嵌入，据称可使语料缩小 40 倍、向量相关性提高 2.3 倍。
边界与风险 这些提升缺少数据集、基线和评测方法，不能直接泛化。持续同步仍需处理删除、权限、版本冲突和失败重试；结构化问答也可能丢失上下文或引入标注偏差。MCP 只是调用协议，不能自动保证召回质量与安全隔离。