Scaling Laws for Grid-Based Approximate Nearest Neighbor Search in High Dimensions

Short summary

Researchers characterize multiprobe grid-based approximate nearest neighbor search on GloVe embeddings, finding constant dimensional scaling while competing methods degrade. The approach achieves near-linear query scaling with lower indexing costs. Since self-attention in transformers can be formalized as ANN operations, these scaling properties may guide efficient transformer architecture design.

• •Grid-based ANN maintains constant dimensional scaling, outperforming graph-, tree-, and partitioning-based methods
• •Achieves near-linear query scaling in dataset size with lower indexing costs
• •Scaling properties inform efficient transformer architecture design

Generated with AI, which can make mistakes.