Research talk: Approximate nearest neighbor search systems at scale

Speaker: Harsha Simhadri, Principal Researcher, Microsoft Research India Building deep learning-based search and recommendation systems at internet scale requires a complete redesign of the search index. Key to this redesign is a fast, accurate, and cost-efficient indexing system for approximate nearest neighbor search. In this talk, we’ll present our recent advances in this space, including the DiskANN and FreshDiskANN systems and the underlying algorithms. These algorithms present an order-of-magnitude improvement in scale and cost-of-operation over the state of the art and are a first of their kind at effectively using solid-state drives (SSDs) to serve at interactive (milliseconds) latencies. In addition, they provide faster in-memory search than other graph indices, like HNSW, and support real-time concurrent insertions and deletions to SSD-resident indices without losing recall. We’ll provide an overview their applicability to various product scenarios and highlight directions for further researc