Kaggle knee-MRI solver hits 0.949 Private LB with Claude Fable labels

The contest's central difficulty was that only 58 of 4,407 training studies carried labels assigned by expert radiologists who read the images. The remaining studies came with radiology report text alone, and those reports follow different criteria from the expert labels used for scoring. Early in the competition, the entrant measured that simply transcribing report statements into labels misclassified about a quarter of the truly positive cases, which pointed to label quality as the main lever for improving the image model.

The solution's key move was to stop asking the LLM what the report said and instead ask it to predict what a panel of radiologists would decide from the images. This reframing raised the label's macro AUC on the 58 expert-labeled studies to 0.9017, compared with 0.8816 for the previous best label set. The final image model, trained only on images, never sees reports at test time, but the quality of its report-derived training labels shaped its performance.

The gap to the top teams appears to lie in what the LLM was given to read. The top three teams had LLMs read both reports and MRI images to build labels, while this solution used reports only. The entrant's own brief test of having an LLM read images was cut short after fewer than 20 minutes on 58 studies, a decision that in hindsight may have cost the top placements.