Apple's SupportNet routes search 1.9× faster than centroid routing

The paper starts from a mismatch that generic vector-search indexes tend to ignore: queries in RAG are short user questions, while the keys they search are long, curated passages. Most approximate methods build their index from the geometry of the keys alone, assuming queries will resemble them. Apple's team instead samples queries from the distribution it expects in production, computes the ground-truth best match once by exhaustive search, and trains small networks to reproduce that mapping — an upfront cost the paper calls amortization.

The two architectures address different bottlenecks. SupportNet replaces centroid routing inside an inverted file index, learning a support function per cluster so a query is sent based on how well its best key could match, not just its distance to a centroid. KeyNet keeps the existing index and instead transforms the query before search, which the paper shows pays off even after accounting for the preprocessing overhead.

Apple positions the choice as practical: KeyNet when inference speed is the constraint, SupportNet when a calibrated score is needed or compute is tight at small model sizes. For very large key sets, the paper suggests combining both — a SupportNet router over all clusters, then a KeyNet per cluster.