Apple TV search gets smarter with personalized ranking from watch history

Apple TV's search experience becomes faster and more accurate when users have a watch history for the system to learn from. Researchers at Apple built a hybrid ranking system that addresses a specific problem: after each keystroke, the system must rank videos even when the user has typed only 1–3 characters and their intent is not yet clear.

The system learns two complementary embeddings. TextEmb is a multilingual text encoder trained via contrastive learning on co-engagement triplets—pairs of videos users watched together—so that similar content lives near each other in the embedding space regardless of language. IdEmb is a collaborative embedding trained purely on what users have watched and clicked, with no text involved. At search time, the system takes the user's recent watch history, generates a representation of their preferences from both models, computes similarity scores between that user representation and each candidate video, and feeds those scores into an XGBoost ranker alongside other features.

Offline evaluation on temporally held-out data showed clear wins. For sessions where the user had watch history, NDCG@10 improved by 2.99% and MRR by 3.30% over the non-personalized baseline. But the slice analysis revealed where the real value lies: on very short queries (1–3 characters), NDCG@10 improved by +8.63%, while on longer queries (where intent is clearer) the gain was only +1.46%. Users with deeper watch histories benefited more; those with 51–100 prior items saw a +4.37% NDCG lift, compared to +2.13% for users with only 1–5 items. Notably, the system improved performance precisely in the cohorts where the default ranker was weakest—long-history users had lower baseline NDCG@10 (0.680 versus 0.733 for new users), yet personalization lifted them more, showing that adding user signals helps most when the text-only approach struggles.

A three-week online controlled experiment validated the offline gains. The treatment group saw a statistically significant +1.14% tap-through rate and +1.23% conversion rate. The converted-item rank position improved by 2.91%, meaning users found what they wanted and tapped it in a higher position in the list—a sign that ranking quality truly improved. The researchers further validated their embeddings by ablating each signal separately (to measure the coverage–precision trade-off between text and collaborative signals) and by using LLM-judged similarity labels on a held-out corpus to evaluate embedding quality while reducing bias from click data.

Apple's personalization approach addresses a core challenge in search: when users type only a few characters, intent is genuinely ambiguous, and traditional text matching alone struggles to predict what they want. By combining text embeddings (semantic understanding of what the query might mean) with collaborative embeddings (what this specific user has historically engaged with), the system can hedge its bets—it learns from both the query itself and the user's past behavior.

The offline analysis reveals the true power of this hybrid approach: the largest gains occur where the baseline system performs worst. Users with rich watch histories and short-prefix queries see NDCG lifts of +8.63%, because in those moments the ranker has the least confidence. Conversely, longer queries and users with sparse history benefit less, because the baseline already knows what to rank. This pattern—that personalization yields the highest ROI where default ranking is weakest—is precisely when adding user signals is most defensible and effective.

No comments yet. Be the first to share your thoughts!