Researchers investigate whether LLMs genuinely learn molecular properties in-context or simply memorize training data through blinded experiments across GPT and Gemini models.

Study tests nine LLM variants across three families (GPT-4.1, GPT-5, Gemini 2.5) to distinguish between genuine in-context learning and memorization in molecular property prediction

Uses systematic blinding approach that progressively removes available information to analyze how pre-trained knowledge interacts with in-context examples

Evaluates models on three MoleculeNet datasets including Delaney solubility, Lipophilicity, and QM7 atomization energy with varying in-context sample sizes

Addresses concern that training data contamination in popular benchmarks may artificially inflate LLM performance on scientific prediction tasks