Optimizing RAG Retrieval with Learned Neural Routers
Apple researchers have introduced an approach to accelerate Maximum Inner Product Search (MIPS) in RAG systems using small, task-specific neural networks. By training models (SupportNet and KeyNet) on query distributions, systems can achieve higher recall at lower latency compared to traditional centroid-based partitioning.