New reinforcement learning approach outperforms k-NN for selecting visual demonstrations in multimodal AI models

Researchers propose Learning to Select Demonstrations (LSD), a reinforcement learning agent that optimizes demonstration selection for multimodal large language models

LSD uses a Dueling DQN with query-centric Transformer Decoder to construct optimal demonstration sets, addressing limitations of traditional unsupervised k-NN similarity search

Testing across five visual regression benchmarks reveals LSD significantly outperforms k-NN on objective, factual regression tasks by avoiding redundant examples

Study identifies a key distinction: k-NN remains better for subjective preference tasks, while LSD excels at capturing full output ranges in complex factual tasks