Researchers develop AI pipeline to automatically annotate sign language videos, addressing critical data shortage in ASL interpretation systems

New pseudo-annotation pipeline processes signed video and English text to generate ranked annotations for glosses, fingerspelled words, and sign classifiers with time intervals

Achieves state-of-the-art fingerspelling recognition on FSBoard dataset (6.7% character error rate) and isolated sign recognition on ASL Citizen (74% top-1 accuracy)

Uses K-Shot LLM approach combined with sparse predictions from fingerspelling and isolated sign recognizers to automate annotations

Targets underutilized professional datasets like ASL STEM Wiki and FLEURS-ASL containing hundreds of hours of video that remain only partially annotated due to prohibitive annotation costs