New PilotBench benchmark reveals that LLMs struggle with safety-critical aviation predictions despite advanced capabilities

PilotBench evaluates 41 LLMs on safety-critical flight trajectory and attitude prediction using 708 real-world general aviation trajectories

Dataset includes synchronized 34-channel telemetry across nine distinct flight phases covering operationally diverse scenarios

Pilot-Score metric balances 60% regression accuracy with 40% instruction adherence and safety compliance to measure model performance

Study uncovers a Precision-Controllability Dichotomy showing traditional forecasters outperform LLMs on physics-governed predictions despite LLMs' semantic understanding

Highlights fundamental challenges in deploying text-trained LLMs as embodied AI agents in safety-critical physical environments