正在打开每日关注
正在打开每日关注
Exa分享索引规模、检索流量与智能体搜索基础设施的增长预判。
Exa is now one of the largest indexes in the world. We serve 80B pages, track 1.4T urls, and are on track to be Google-scale in early 2027. It's difficult to know the size of indexes, but we estimate Google to be ~1T, Bing to be ~500B, and Yandex ~200B. Brave mentioned in April that they're at 40B. Most of the web is trash that can hurt AI outputs, so you actually have to crawl way more than you serve and train models to filter out the trash. In terms of peak QPS, we estimate Bing at roughly 30k/s and Google at ~500k/s. In our case, because agent traffic often requires large fan-out (deep searches can use dozens of sub-searches) and can be spiky (like when AI labs RL with us), we're already starting to provision for Bing-scale traffic. Within 2 years, agents will search at many millions per second. They'll also want comprehensiveness over all data, far bigger than any index today. So the retrieval infra required will be larger than Google-scale in both dimensions.
是什么 这是一条关于搜索索引与智能体检索基础设施的行业观点。Exa称其已服务约800亿网页、追踪1.4万亿个URL,并预计在2027年初达到接近Google的规模。推文同时给出Google、Bing、Yandex和Brave的索引与峰值QPS估算,用于说明竞争格局和基础设施量级。
核心洞察 索引规模并不等于有效数据规模。由于网络内容中存在大量低质量信息,搜索服务必须先大规模抓取,再依靠模型和过滤系统筛选可供AI使用的内容。智能体搜索还会产生明显不同于传统搜索的负载:一次任务可能展开几十个子查询,AI实验又会制造突发流量,因此扇出、弹性和峰值容量比平均请求量更关键。
关键意义 随着智能体从简单问答转向全面研究,未来搜索请求可能达到每秒数百万次,对索引覆盖、并发检索、排序过滤和成本控制提出更高要求。Exa的判断意味着检索层可能成为AI基础设施中的核心瓶颈。
边界与风险 所有规模、QPS和时间表均为推文作者的估算或预测,缺少统一统计口径与第三方验证;“Google规模”也可能只指索引或流量的某一维度,不能直接视为整体能力对标。